Jul 09 04:00:00.630821 ip-10-0-135-255 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 04:00:00.942532 ip-10-0-135-255 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 04:00:00.942532 ip-10-0-135-255 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 04:00:00.942532 ip-10-0-135-255 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 04:00:00.942532 ip-10-0-135-255 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 04:00:00.942532 ip-10-0-135-255 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 04:00:00.943782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.943692    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 04:00:00.951071 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951044    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.951071 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951066    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.951071 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951070    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.951071 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951074    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951078    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951081    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951084    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951087    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951090    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951093    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951096    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951100    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951103    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951106    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951108    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951111    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951114    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951116    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951119    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951121    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951124    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951126    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951129    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.951192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951132    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951135    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951138    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951141    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951143    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951146    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951148    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951151    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951154    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951156    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951159    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951161    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951164    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951166    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951169    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951173    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951176    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951179    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951181    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.951677 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951184    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951187    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951189    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951195    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951199    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951202    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951206    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951210    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951213    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951216    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951219    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951222    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951225    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951228    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951231    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951234    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951237    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951239    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951242    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.952136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951245    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951248    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951250    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951253    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951257    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951260    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951263    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951265    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951268    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951271    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951274    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951277    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951280    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951282    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951285    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951287    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951290    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951293    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951295    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951298    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.952617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951301    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951304    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951306    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951310    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951313    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951316    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951320    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951324    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951327    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951330    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951333    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951336    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951338    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951341    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951344    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951346    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951349    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.951352    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952185    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952193    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.953095 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952197    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952201    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952204    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952208    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952211    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952214    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952217    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952219    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952222    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952225    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952227    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952230    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952233    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952237    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952241    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952243    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952246    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952249    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952251    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.953586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952254    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952257    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952260    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952262    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952265    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952268    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952270    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952273    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952276    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952278    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952281    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952284    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952286    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952289    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952292    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952296    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952299    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952302    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952304    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952307    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.954104 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952310    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952312    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952315    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952318    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952320    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952322    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952325    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952328    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952330    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952333    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952335    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952338    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952341    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952345    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952348    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952352    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952354    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952357    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952360    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.954629 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952363    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952365    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952368    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952371    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952373    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952376    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952378    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952381    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952384    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952387    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952389    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952398    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952400    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952403    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952405    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952408    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952410    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952413    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952415    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952418    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.955094 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952420    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952424    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952426    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952429    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952431    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952433    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952436    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952439    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952441    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952444    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952446    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952449    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952452    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952454    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952459    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952462    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952482    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952485    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.952488    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953489    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.955586 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953496    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953499    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953503    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953506    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953509    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953512    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953514    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953517    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953520    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953522    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953525    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953527    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953529    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953532    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953534    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953537    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953540    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953543    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953546    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953548    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.956076 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953551    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953554    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953558    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953562    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953565    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953568    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953571    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953574    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953577    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953579    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953582    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953585    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953588    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953591    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953594    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953597    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953600    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953603    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953606    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.956570 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953608    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953612    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953616    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953619    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953622    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953624    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953627    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953629    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953632    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953634    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953637    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953639    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953642    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953645    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953647    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953650    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953652    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953655    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953657    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953660    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.957023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953663    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953665    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953668    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953670    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953673    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953676    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953678    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953680    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953683    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953685    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953688    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953691    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953693    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953696    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953698    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953701    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953704    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953706    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953708    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953711    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.957537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953714    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953716    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953719    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953721    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953723    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953726    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953728    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953733    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953737    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953740    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953742    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953745    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953748    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953751    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953753    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953756    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953758    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953761    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.953764    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.958004 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953843    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953850    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953857    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953862    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953867    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953871    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953875    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953880    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953883    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953886    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953889    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953893    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953896    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953899    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953902    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953905    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953908    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953911    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953915    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953918    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953922    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953925    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953928    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953932    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 04:00:00.958488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953935    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953939    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953942    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953945    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953948    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953952    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953955    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953958    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953962    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953965    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953968    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953971    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953974    2577 flags.go:64] FLAG: --enable-server="true"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953978    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953982    2577 flags.go:64] FLAG: --event-burst="100"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953986    2577 flags.go:64] FLAG: --event-qps="50"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953989    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953992    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953995    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.953999    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954002    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954005    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954008    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954011    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954014    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 04:00:00.959093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954017    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954020    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954023    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954029    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954031    2577 flags.go:64] FLAG: --feature-gates=""
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954035    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954038    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954042    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954045    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954048    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954051    2577 flags.go:64] FLAG: --help="false"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954055    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-255.ec2.internal"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954058    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954061    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954064    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954067    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954071    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954073    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954076    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954079    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954082    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954086    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954089    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954092    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 09 04:00:00.959733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954095    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954097    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954101    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954103    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954106    2577 flags.go:64] FLAG: --lock-file=""
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954109    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954112    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954116    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954122    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954125    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954127    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954133    2577 flags.go:64] FLAG: --logging-format="text"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954136    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954139    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954143    2577 flags.go:64] FLAG: --manifest-url=""
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954145    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954152    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954155    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954159    2577 flags.go:64] FLAG: --max-pods="110"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954162    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954165    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954175    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954180    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954184    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954187    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 04:00:00.960331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954190    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954198    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954201    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954204    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954207    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954211    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954217    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954220    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954223    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954226    2577 flags.go:64] FLAG: --port="10250"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954229    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954232    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f05a1babd758867d"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954235    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954238    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954241    2577 flags.go:64] FLAG: --register-node="true"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954244    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954248    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954251    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954254    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954258    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954262    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954265    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954268    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954272    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954276    2577 flags.go:64] FLAG: --runonce="false"
Jul 09 04:00:00.960943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954278    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954281    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954284    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954287    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954290    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954293    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954296    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954299    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954302    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954305    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954308    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954311    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954314    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954325    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954328    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954333    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954336    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954339    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954344    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954347    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954350    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954353    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954356    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954359    2577 flags.go:64] FLAG: --v="2"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954368    2577 flags.go:64] FLAG: --version="false"
Jul 09 04:00:00.961577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954372    2577 flags.go:64] FLAG: --vmodule=""
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954377    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.954380    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954910    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954915    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954920    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954924    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954928    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954930    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954933    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954936    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954939    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954942    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954946    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954949    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954952    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954955    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954958    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954960    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954963    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.962228 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954966    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954970    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954973    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954976    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954979    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954981    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954986    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954990    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954994    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954997    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.954999    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955002    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955005    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955010    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955012    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955015    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955018    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955022    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.962737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955024    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955027    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955030    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955034    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955038    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955042    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955045    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955049    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955053    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955057    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955060    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955064    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955068    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955071    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955075    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955079    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955083    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955087    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955089    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955092    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.963186 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955095    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955098    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955101    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955104    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955106    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955109    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955112    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955116    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955119    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955121    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955124    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955128    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955131    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955133    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955136    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955139    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955141    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955144    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955147    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955150    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.963700 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955152    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955155    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955158    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955160    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955163    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955165    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955168    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955170    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955175    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955179    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955181    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955184    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955186    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955189    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955191    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955194    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955196    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955199    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955201    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955205    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.964192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955208    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955211    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955213    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.955217    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.955223    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.961961    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.961982    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.962000    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.962010    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962073    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962079    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962082    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962086    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962089    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962092    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.964692 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962096    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962099    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962102    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962105    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962107    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962110    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962113    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962116    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962119    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962122    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962124    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962127    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962130    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962133    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962136    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962138    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962140    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962143    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962146    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962148    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.965097 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962151    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962154    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962156    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962161    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962163    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962174    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962180    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962185    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962188    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962191    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962194    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962198    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962201    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962204    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962207    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962210    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962212    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962215    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962219    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.965605 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962223    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962226    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962229    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962233    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962237    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962241    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962244    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962247    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962250    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962252    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962255    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962257    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962260    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962263    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962265    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962268    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962271    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962275    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962277    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.966083 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962280    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962283    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962285    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962288    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962290    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962293    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962296    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962298    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962301    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962303    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962305    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962308    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962311    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962313    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962316    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962318    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962321    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962324    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962326    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962329    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.966559 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962331    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962334    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962337    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962340    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962342    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962345    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962347    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962350    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962353    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962356    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962359    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962362    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962365    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962367    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962370    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.967054 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.962375    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962520    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962527    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962530    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962533    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962536    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962539    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962541    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962544    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962547    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962549    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962552    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962555    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962557    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962560    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962563    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962565    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962568    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962570    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962573    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 04:00:00.967444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962576    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962578    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962581    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962584    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962586    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962589    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962591    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962594    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962597    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962600    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962602    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962606    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962610    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962614    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962616    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962619    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962623    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962625    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962628    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 04:00:00.968137 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962630    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962634    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962637    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962640    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962643    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962646    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962649    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962651    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962654    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962657    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962660    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962662    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962665    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962668    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962671    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962673    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962676    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962678    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962681    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962684    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 04:00:00.968621 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962687    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962689    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962692    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962695    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962697    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962700    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962703    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962705    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962707    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962710    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962712    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962715    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962718    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962738    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962742    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962746    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962749    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962752    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962754    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962757    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 04:00:00.969117 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962759    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962762    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962764    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962767    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962770    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962773    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962775    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962778    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962780    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962784    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962788    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962790    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962793    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962796    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962798    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962801    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962804    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962806    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962809    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962812    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 04:00:00.969767 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:00.962814    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.962819    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.962961    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.964945    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.966081    2577 server.go:1019] "Starting client certificate rotation"
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.966206    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.966244    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 04:00:00.970685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.968171    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 04:00:00.971510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.971462    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 04:00:00.979336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.979318    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 09 04:00:00.981410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.981392    2577 log.go:25] "Validated CRI v1 image API"
Jul 09 04:00:00.982331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.982302    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 04:00:00.986108 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.986079    2577 fs.go:135] Filesystem UUIDs: map[49165438-a5d0-48e8-a425-243306db7f57:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f75a385c-5216-4d35-b788-68fb8fbf4fc3:/dev/nvme0n1p3]
Jul 09 04:00:00.986176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.986106    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 04:00:00.991932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.991803    2577 manager.go:216] Machine: {Timestamp:2026-07-09 04:00:00.990590737 +0000 UTC m=+0.264968639 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100083 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24eb7a4c50a71cb18980be885eba89 SystemUUID:ec24eb7a-4c50-a71c-b189-80be885eba89 BootID:361d0e9e-7c13-48f4-b667-793650ccc2cd Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4d:40:f2:22:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4d:40:f2:22:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:5b:7d:5f:81:72 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 04:00:00.991932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.991927    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 04:00:00.992064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.992020    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 04:00:00.994634 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.994593    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 04:00:00.994791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.994640    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-255.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 04:00:00.995346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.995335    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 04:00:00.995385 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.995349    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 04:00:00.995385 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.995357    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 04:00:00.995870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.995858    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 04:00:00.997095 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.997077    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 04:00:00.997193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:00.997181    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 04:00:01.001161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001147    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 04:00:01.001488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001477    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 04:00:01.001532 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001514    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 04:00:01.001532 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001526    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 04:00:01.001602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001553    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 04:00:01.001602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001564    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 09 04:00:01.001602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001574    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 04:00:01.001945 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001934    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 04:00:01.001991 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.001952    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 04:00:01.003891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.003851    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 04:00:01.006060 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.006042    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 04:00:01.006116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.006078    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 04:00:01.007195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007183    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007200    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007207    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007212    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007222    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007228    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 04:00:01.007236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007237    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 04:00:01.007446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007243    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 04:00:01.007446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007254    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 04:00:01.007446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007260    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 04:00:01.007446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007268    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 04:00:01.007446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007274    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 04:00:01.007928 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007916    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 04:00:01.007928 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.007927    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 04:00:01.012112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.012003    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-255.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 04:00:01.012112 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.012073    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 04:00:01.012610 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.012592    2577 server.go:1268] "Started kubelet"
Jul 09 04:00:01.012751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.012719    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 04:00:01.012836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.012784    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 04:00:01.012913 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.012890    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 04:00:01.013737 ip-10-0-135-255 systemd[1]: Started Kubernetes Kubelet.
Jul 09 04:00:01.014049 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.014021    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 04:00:01.014739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.014724    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 04:00:01.019008 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.018985    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 04:00:01.019367 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.019348    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 04:00:01.020048 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.019294    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-255.ec2.internal.18c0831a3488620d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-255.ec2.internal,UID:ip-10-0-135-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-255.ec2.internal,},FirstTimestamp:2026-07-09 04:00:01.012457997 +0000 UTC m=+0.286835899,LastTimestamp:2026-07-09 04:00:01.012457997 +0000 UTC m=+0.286835899,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-255.ec2.internal,}"
Jul 09 04:00:01.021231 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.021213    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 04:00:01.021231 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.021223    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 04:00:01.021359 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.021326    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.021860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.021736    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 04:00:01.022046 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022030    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 04:00:01.022179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022149    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 04:00:01.022286 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022181    2577 factory.go:55] Registering systemd factory
Jul 09 04:00:01.022286 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022232    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 09 04:00:01.022407 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022309    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 04:00:01.022407 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022318    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 04:00:01.022514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022496    2577 factory.go:153] Registering CRI-O factory
Jul 09 04:00:01.022514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022516    2577 factory.go:223] Registration of the crio container factory successfully
Jul 09 04:00:01.022685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022575    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 04:00:01.022685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022600    2577 factory.go:103] Registering Raw factory
Jul 09 04:00:01.022685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.022617    2577 manager.go:1196] Started watching for new ooms in manager
Jul 09 04:00:01.023320 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.023301    2577 manager.go:349] Starting recovery of all containers
Jul 09 04:00:01.024559 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.024514    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 04:00:01.024660 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.024621    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 04:00:01.029371 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.029172    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjnkn"
Jul 09 04:00:01.030761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.030728    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 04:00:01.035151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.035128    2577 manager.go:354] Recovery completed
Jul 09 04:00:01.039007 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.038002    2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 09 04:00:01.039007 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.038687    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjnkn"
Jul 09 04:00:01.042147 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.042133    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.045767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.045745    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.045853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.045786    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.045853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.045803    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.046516 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.046499    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 04:00:01.046605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.046539    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.046605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.046542    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 04:00:01.046686 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.046617    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 04:00:01.047324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.047308    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.047368 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.047339    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.047368 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.047351    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.048474 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.048388    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-255.ec2.internal.18c0831a3684a188  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-255.ec2.internal,UID:ip-10-0-135-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-255.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-255.ec2.internal,},FirstTimestamp:2026-07-09 04:00:01.045766536 +0000 UTC m=+0.320144442,LastTimestamp:2026-07-09 04:00:01.045766536 +0000 UTC m=+0.320144442,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-255.ec2.internal,}"
Jul 09 04:00:01.048805 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.048791    2577 policy_none.go:49] "None policy: Start"
Jul 09 04:00:01.048843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.048810    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 04:00:01.048843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.048822    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 04:00:01.050950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.050938    2577 policy_none.go:47] "Start"
Jul 09 04:00:01.092176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092158    2577 manager.go:337] "Starting Device Plugin manager"
Jul 09 04:00:01.092303 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.092194    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 04:00:01.092303 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092204    2577 server.go:87] "Starting device plugin registration server"
Jul 09 04:00:01.092512 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092497    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 04:00:01.092592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092531    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 04:00:01.092643 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092624    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 04:00:01.092741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092720    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 04:00:01.092741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.092734    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 04:00:01.093268 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.093251    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 04:00:01.093356 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.093306    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.173042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.173008    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 04:00:01.175466 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.173066    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 04:00:01.175466 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.173098    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 04:00:01.175466 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.173148    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 04:00:01.175807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.175787    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:01.193442 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.193359    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.194723 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.194704    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.194817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.194738    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.194817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.194749    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.194817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.194775    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.204016 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.203993    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.204076 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.204024    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-255.ec2.internal\": node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.222709 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.222679    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.275142 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.275081    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"]
Jul 09 04:00:01.275324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.275184    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.276966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.276950    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.277046 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.276982    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.277046 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.276998    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.279604 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.279590    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.279736 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.279718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.279790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.279753    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.281794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281776    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.281880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281807    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.281880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281817    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.281880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281780    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.281988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281888    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.281988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.281903    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.284290 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.284276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.284347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.284302    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 04:00:01.285216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.285197    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 04:00:01.285310 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.285233    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 04:00:01.285310 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.285248    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeHasSufficientPID"
Jul 09 04:00:01.314664 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.314628    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-255.ec2.internal\" not found" node="ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.319576 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.319531    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-255.ec2.internal\" not found" node="ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.323080 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.323060    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.325384 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.325367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed53264ef20f88c821b678c86f72e5cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-255.ec2.internal\" (UID: \"ed53264ef20f88c821b678c86f72e5cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.325456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.325392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.325456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.325412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.423640 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.423607    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.425999 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.425974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed53264ef20f88c821b678c86f72e5cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-255.ec2.internal\" (UID: \"ed53264ef20f88c821b678c86f72e5cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.426103 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.426014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.426103 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.426040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.426103 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.426092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.426234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.426115    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9c32fa5fb84f7336ef4d31ff1741989-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal\" (UID: \"c9c32fa5fb84f7336ef4d31ff1741989\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.426234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.426092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed53264ef20f88c821b678c86f72e5cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-255.ec2.internal\" (UID: \"ed53264ef20f88c821b678c86f72e5cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.524426 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.524334    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.624157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.624109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.624249 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.624118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.625237 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.625218    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.726017 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.725977    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.827174 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:01.827085    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-255.ec2.internal\" not found"
Jul 09 04:00:01.880681 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.880657    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:01.923578 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.923541    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.935565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.935541    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 04:00:01.937058 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.937043    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal"
Jul 09 04:00:01.940082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.940064    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:01.946055 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.946038    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 04:00:01.966768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.966741    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 04:00:01.966912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.966889    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 04:00:01.966959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.966925    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 04:00:01.966959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:01.966930    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 04:00:02.001860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.001812    2577 apiserver.go:52] "Watching apiserver"
Jul 09 04:00:02.015464 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.015436    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 04:00:02.017405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.016918    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9cr6q","kube-system/konnectivity-agent-g5vfb","kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4","openshift-dns/node-resolver-xbljd","openshift-network-diagnostics/network-check-target-986sz","openshift-network-operator/iptables-alerter-24cd2","openshift-cluster-node-tuning-operator/tuned-4nmdx","openshift-image-registry/node-ca-xmpxf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal","openshift-multus/multus-additional-cni-plugins-88f9s","openshift-multus/multus-fltq9","openshift-multus/network-metrics-daemon-st6ww","openshift-ovn-kubernetes/ovnkube-node-jchmt"]
Jul 09 04:00:02.019761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.019733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.021348 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.021326    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 04:00:02.021622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.021604    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.021622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.021614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nssbm\""
Jul 09 04:00:02.021764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.021631    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 04:00:02.021953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.021931    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.022186 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.022171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.022259 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.022240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:02.024647 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.024630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.026318 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.026301    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.027086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.027071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.028425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qf5n\" (UniqueName: \"kubernetes.io/projected/87958160-8026-4072-98a4-fe3b1faf6302-kube-api-access-7qf5n\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.028511 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.028511 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6bms\" (UniqueName: \"kubernetes.io/projected/ba58ffef-ea23-486c-ad2e-be920dc8173c-kube-api-access-c6bms\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.028641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87958160-8026-4072-98a4-fe3b1faf6302-iptables-alerter-script\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.028641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87958160-8026-4072-98a4-fe3b1faf6302-host-slash\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.028641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.028570    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 04:00:02.029091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029077    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4lwdd\""
Jul 09 04:00:02.029159 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029093    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.029159 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029138    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 04:00:02.029263 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029176    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 04:00:02.029426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.029557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.029853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029836    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 04:00:02.029993 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029977    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-85vm9\""
Jul 09 04:00:02.030061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.029976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.030061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.030031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 04:00:02.031323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.031307    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.031524 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.031509    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.031657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.031643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f95pl\""
Jul 09 04:00:02.031997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.031981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:02.032071 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.032051    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:02.034419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.034405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.036084 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.036067    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.036260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.036246    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.036322 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.036290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rcfwm\""
Jul 09 04:00:02.036363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.036291    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 04:00:02.036812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.036797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.036881 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.036862    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:02.037283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.037269    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:02.039235 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.039216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.039952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.039931    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 04:00:02.040946 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.040929    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 04:00:02.041134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.041120    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zn7lw\""
Jul 09 04:00:02.041212 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.041155    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 04:00:02.041786 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.041770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.041934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.041901    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 03:55:01 +0000 UTC" deadline="2028-03-06 18:18:06.675440561 +0000 UTC"
Jul 09 04:00:02.041934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.041933    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14558h18m4.633512319s"
Jul 09 04:00:02.043679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.043461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w6fg4\""
Jul 09 04:00:02.043679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.043498    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.043679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.043467    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.044267 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.044251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.045820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.045802    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 04:00:02.045992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.045959    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 04:00:02.045992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.045971    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 04:00:02.046165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.046014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7l2fp\""
Jul 09 04:00:02.046165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.046123    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 04:00:02.046419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.046397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 04:00:02.046571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.046556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.048047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.048032    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 04:00:02.049148 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.049134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-86lh2\""
Jul 09 04:00:02.059342 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.059301    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7sv55"
Jul 09 04:00:02.066682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.066662    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7sv55"
Jul 09 04:00:02.123624 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.123606    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 04:00:02.128726 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cnibin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.128836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87958160-8026-4072-98a4-fe3b1faf6302-iptables-alerter-script\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.128836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87958160-8026-4072-98a4-fe3b1faf6302-host-slash\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.128967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-kubelet\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.128967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.128967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128926    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87958160-8026-4072-98a4-fe3b1faf6302-host-slash\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.129090 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.128982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129090 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-modprobe-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129090 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-netns\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.129090 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6bms\" (UniqueName: \"kubernetes.io/projected/ba58ffef-ea23-486c-ad2e-be920dc8173c-kube-api-access-c6bms\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-env-overrides\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-dbus\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-lib-modules\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-multus-certs\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-cnibin\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.129250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-system-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qf5n\" (UniqueName: \"kubernetes.io/projected/87958160-8026-4072-98a4-fe3b1faf6302-kube-api-access-7qf5n\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-config\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-script-lib\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxp5g\" (UniqueName: \"kubernetes.io/projected/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-kube-api-access-fxp5g\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-var-lib-kubelet\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mwmb\" (UniqueName: \"kubernetes.io/projected/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-kube-api-access-7mwmb\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-ovn\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.129513    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysconfig\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.129571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrsl7\" (UniqueName: \"kubernetes.io/projected/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kube-api-access-rrsl7\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.129600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:02.629574341 +0000 UTC m=+1.903952258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-systemd-units\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-os-release\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-netd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8d6e6e9-c491-40bf-b405-df699fc273b6-host\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8d6e6e9-c491-40bf-b405-df699fc273b6-serviceca\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-host\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-registration-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.129990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-system-cni-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-kubelet-config\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-tuned\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cni-binary-copy\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-kubelet\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.130151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-device-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knqg7\" (UniqueName: \"kubernetes.io/projected/c8d6e6e9-c491-40bf-b405-df699fc273b6-kube-api-access-knqg7\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-conf\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-socket-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-binary-copy\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-slash\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-log-socket\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-bin\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-hostroot\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-etc-kubernetes\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-sys-fs\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-systemd\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-sys\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.130819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-k8s-cni-cncf-io\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-node-log\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snphg\" (UniqueName: \"kubernetes.io/projected/c1c97c43-619e-492d-8e51-7ec49f54eaea-kube-api-access-snphg\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-var-lib-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-kubernetes\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-multus\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca467583-1e69-4d74-8c16-e5e033f74b6e-konnectivity-ca\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-etc-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29bad720-862e-46d4-a5e6-39b7fe607ff6-hosts-file\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29bad720-862e-46d4-a5e6-39b7fe607ff6-tmp-dir\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca467583-1e69-4d74-8c16-e5e033f74b6e-agent-certs\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-conf-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-os-release\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.130991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.131566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-netns\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovn-node-metrics-cert\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-daemon-config\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lldh2\" (UniqueName: \"kubernetes.io/projected/29bad720-862e-46d4-a5e6-39b7fe607ff6-kube-api-access-lldh2\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-systemd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-run\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-tmp\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-socket-dir-parent\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-bin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5gdb\" (UniqueName: \"kubernetes.io/projected/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-kube-api-access-m5gdb\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.132294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.131406    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87958160-8026-4072-98a4-fe3b1faf6302-iptables-alerter-script\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.135117 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.135098    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 04:00:02.138186 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.138167    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qf5n\" (UniqueName: \"kubernetes.io/projected/87958160-8026-4072-98a4-fe3b1faf6302-kube-api-access-7qf5n\") pod \"iptables-alerter-24cd2\" (UID: \"87958160-8026-4072-98a4-fe3b1faf6302\") " pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.139964 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.139943    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6bms\" (UniqueName: \"kubernetes.io/projected/ba58ffef-ea23-486c-ad2e-be920dc8173c-kube-api-access-c6bms\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.231967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.231920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29bad720-862e-46d4-a5e6-39b7fe607ff6-hosts-file\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.231967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.231964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29bad720-862e-46d4-a5e6-39b7fe607ff6-tmp-dir\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232041    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29bad720-862e-46d4-a5e6-39b7fe607ff6-hosts-file\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca467583-1e69-4d74-8c16-e5e033f74b6e-agent-certs\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-conf-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-os-release\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-netns\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovn-node-metrics-cert\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-daemon-config\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232227    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-os-release\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lldh2\" (UniqueName: \"kubernetes.io/projected/29bad720-862e-46d4-a5e6-39b7fe607ff6-kube-api-access-lldh2\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-systemd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232283    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-conf-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-run\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-tmp\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232330    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29bad720-862e-46d4-a5e6-39b7fe607ff6-tmp-dir\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-socket-dir-parent\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232377    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-systemd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232411    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-run\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232420    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-socket-dir-parent\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-bin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232440    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-bin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5gdb\" (UniqueName: \"kubernetes.io/projected/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-kube-api-access-m5gdb\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cnibin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-kubelet\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.232708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-modprobe-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-netns\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-env-overrides\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-dbus\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-lib-modules\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-multus-certs\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-cnibin\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-system-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232808    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-config\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232871    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-system-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232338    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-netns\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-script-lib\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxp5g\" (UniqueName: \"kubernetes.io/projected/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-kube-api-access-fxp5g\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-var-lib-kubelet\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mwmb\" (UniqueName: \"kubernetes.io/projected/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-kube-api-access-7mwmb\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.232993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.233650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-ovn\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-daemon-config\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysconfig\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrsl7\" (UniqueName: \"kubernetes.io/projected/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kube-api-access-rrsl7\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-systemd-units\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233169    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cnibin\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-os-release\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-netd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8d6e6e9-c491-40bf-b405-df699fc273b6-host\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8d6e6e9-c491-40bf-b405-df699fc273b6-serviceca\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-host\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-registration-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-system-cni-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233397    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.234527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-kubelet-config\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-tuned\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233505    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-system-cni-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233513    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-config\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233577    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-var-lib-kubelet\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233577    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-registration-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233602    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-host\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233520    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8d6e6e9-c491-40bf-b405-df699fc273b6-host\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233674    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-dbus\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233719    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysconfig\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233726    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dcb5037-c133-44cf-9617-ec0a07a55301-kubelet-config\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233825    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-lib-modules\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-run-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233863    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-multus-certs\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233876    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-systemd-units\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233893    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233912    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-kubelet\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.235357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233938    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-run-ovn\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233977    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.233989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-modprobe-d\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cni-binary-copy\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8d6e6e9-c491-40bf-b405-df699fc273b6-serviceca\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234037    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1c97c43-619e-492d-8e51-7ec49f54eaea-cnibin\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-kubelet\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-device-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knqg7\" (UniqueName: \"kubernetes.io/projected/c8d6e6e9-c491-40bf-b405-df699fc273b6-kube-api-access-knqg7\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-conf\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-socket-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-binary-copy\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-slash\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-log-socket\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-bin\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.236228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-hostroot\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-etc-kubernetes\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234603    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-cni-binary-copy\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-sys-fs\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234735    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovnkube-script-lib\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234774    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-sys-fs\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-systemd\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-sys\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.234991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-k8s-cni-cncf-io\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-node-log\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snphg\" (UniqueName: \"kubernetes.io/projected/c1c97c43-619e-492d-8e51-7ec49f54eaea-kube-api-access-snphg\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-var-lib-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-kubernetes\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-multus\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.236878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca467583-1e69-4d74-8c16-e5e033f74b6e-konnectivity-ca\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-etc-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235205    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-tmp\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-binary-copy\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-ovn-node-metrics-cert\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-sys\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-k8s-cni-cncf-io\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235332    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-os-release\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235341    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-netd\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-hostroot\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235377    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-kubelet\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235399    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235416    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-run-netns\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.235427    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235439    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-etc-kubernetes\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.235488    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:02.735465326 +0000 UTC m=+2.009843214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235513    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-socket-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.237395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-systemd\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235678    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-log-socket\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-etc-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235722    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-device-dir\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235757    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-multus-cni-dir\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235797    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-node-log\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235963    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-slash\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.235978    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1c97c43-619e-492d-8e51-7ec49f54eaea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236030    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-var-lib-openvswitch\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236066    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-host-cni-bin\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236104    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-sysctl-conf\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236122    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-kubernetes\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236174    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-host-var-lib-cni-multus\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-env-overrides\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-etc-tuned\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236580    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca467583-1e69-4d74-8c16-e5e033f74b6e-konnectivity-ca\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.237952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.236879    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca467583-1e69-4d74-8c16-e5e033f74b6e-agent-certs\") pod \"konnectivity-agent-g5vfb\" (UID: \"ca467583-1e69-4d74-8c16-e5e033f74b6e\") " pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.239516 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.239495    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:02.239655 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.239521    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:02.239655 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.239535    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:02.239655 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.239640    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:02.739621922 +0000 UTC m=+2.013999813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:02.242079 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.242046    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lldh2\" (UniqueName: \"kubernetes.io/projected/29bad720-862e-46d4-a5e6-39b7fe607ff6-kube-api-access-lldh2\") pod \"node-resolver-xbljd\" (UID: \"29bad720-862e-46d4-a5e6-39b7fe607ff6\") " pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.242393 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.242376    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrsl7\" (UniqueName: \"kubernetes.io/projected/5feb688e-54d8-4789-b86b-75f5b0a3b1b2-kube-api-access-rrsl7\") pod \"aws-ebs-csi-driver-node-tt4s4\" (UID: \"5feb688e-54d8-4789-b86b-75f5b0a3b1b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.242485 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.242464    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5gdb\" (UniqueName: \"kubernetes.io/projected/de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a-kube-api-access-m5gdb\") pod \"multus-fltq9\" (UID: \"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a\") " pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.242657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.242640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxp5g\" (UniqueName: \"kubernetes.io/projected/83bbe97f-cc76-4bb6-81f6-a8abbe5ada48-kube-api-access-fxp5g\") pod \"ovnkube-node-jchmt\" (UID: \"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48\") " pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.242957 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.242929    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mwmb\" (UniqueName: \"kubernetes.io/projected/98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f-kube-api-access-7mwmb\") pod \"tuned-4nmdx\" (UID: \"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f\") " pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.245084 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.245055    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snphg\" (UniqueName: \"kubernetes.io/projected/c1c97c43-619e-492d-8e51-7ec49f54eaea-kube-api-access-snphg\") pod \"multus-additional-cni-plugins-88f9s\" (UID: \"c1c97c43-619e-492d-8e51-7ec49f54eaea\") " pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.245315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.245297    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knqg7\" (UniqueName: \"kubernetes.io/projected/c8d6e6e9-c491-40bf-b405-df699fc273b6-kube-api-access-knqg7\") pod \"node-ca-xmpxf\" (UID: \"c8d6e6e9-c491-40bf-b405-df699fc273b6\") " pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.360906 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.360804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-24cd2"
Jul 09 04:00:02.366820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.366795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:02.398098 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.398054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4"
Jul 09 04:00:02.400971 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.400953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xbljd"
Jul 09 04:00:02.430302 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.430264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xmpxf"
Jul 09 04:00:02.445829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.445798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:02.450950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.450923    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4nmdx"
Jul 09 04:00:02.478068 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.478030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-88f9s"
Jul 09 04:00:02.483209 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.483189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fltq9"
Jul 09 04:00:02.639827 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.639729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:02.639980 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.639848    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:02.639980 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.639922    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:03.639905662 +0000 UTC m=+2.914283549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:02.740909 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.740867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:02.741132 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.740927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:02.741132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741029    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:02.741132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741057    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:02.741132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741068    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:02.741132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741034    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:02.741330 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741125    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:03.741104988 +0000 UTC m=+3.015482894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:02.741330 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:02.741189    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:03.741173121 +0000 UTC m=+3.015551010 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:02.885893 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:02.885855    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c32fa5fb84f7336ef4d31ff1741989.slice/crio-8c3755abd22743c5117a2d6740fe978dcfc1971e950757845cdaa5c29f78628a WatchSource:0}: Error finding container 8c3755abd22743c5117a2d6740fe978dcfc1971e950757845cdaa5c29f78628a: Status 404 returned error can't find the container with id 8c3755abd22743c5117a2d6740fe978dcfc1971e950757845cdaa5c29f78628a
Jul 09 04:00:02.888110 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:02.888086    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded53264ef20f88c821b678c86f72e5cd.slice/crio-673645ee453ec6919ed5c0487a7b704dcf9fe6b7556e6abe576d19779b680e6b WatchSource:0}: Error finding container 673645ee453ec6919ed5c0487a7b704dcf9fe6b7556e6abe576d19779b680e6b: Status 404 returned error can't find the container with id 673645ee453ec6919ed5c0487a7b704dcf9fe6b7556e6abe576d19779b680e6b
Jul 09 04:00:02.892167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:02.892141    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:00:03.070288 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.070243    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:55:02 +0000 UTC" deadline="2027-12-11 07:10:36.924375866 +0000 UTC"
Jul 09 04:00:03.070288 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.070282    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12483h10m33.854098746s"
Jul 09 04:00:03.082184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.082158    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:03.089726 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.089698    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d6e6e9_c491_40bf_b405_df699fc273b6.slice/crio-b18511119668a1a3f1ca2dc71882aa973f36fac98d8b7ac44685cef43ada3435 WatchSource:0}: Error finding container b18511119668a1a3f1ca2dc71882aa973f36fac98d8b7ac44685cef43ada3435: Status 404 returned error can't find the container with id b18511119668a1a3f1ca2dc71882aa973f36fac98d8b7ac44685cef43ada3435
Jul 09 04:00:03.166601 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.166559    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1a7fdc_d6bf_4bdd_8c62_f9239d56c64a.slice/crio-be38bc09c95ff9f8775e34d70eb7de915e52fc32fc75e7bb56a447a7f7a30a97 WatchSource:0}: Error finding container be38bc09c95ff9f8775e34d70eb7de915e52fc32fc75e7bb56a447a7f7a30a97: Status 404 returned error can't find the container with id be38bc09c95ff9f8775e34d70eb7de915e52fc32fc75e7bb56a447a7f7a30a97
Jul 09 04:00:03.173367 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.173342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:03.173523 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.173472    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:03.177441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.177395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fltq9" event={"ID":"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a","Type":"ContainerStarted","Data":"be38bc09c95ff9f8775e34d70eb7de915e52fc32fc75e7bb56a447a7f7a30a97"}
Jul 09 04:00:03.178638 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.178617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xmpxf" event={"ID":"c8d6e6e9-c491-40bf-b405-df699fc273b6","Type":"ContainerStarted","Data":"b18511119668a1a3f1ca2dc71882aa973f36fac98d8b7ac44685cef43ada3435"}
Jul 09 04:00:03.179700 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.179679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal" event={"ID":"ed53264ef20f88c821b678c86f72e5cd","Type":"ContainerStarted","Data":"673645ee453ec6919ed5c0487a7b704dcf9fe6b7556e6abe576d19779b680e6b"}
Jul 09 04:00:03.180762 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.180736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal" event={"ID":"c9c32fa5fb84f7336ef4d31ff1741989","Type":"ContainerStarted","Data":"8c3755abd22743c5117a2d6740fe978dcfc1971e950757845cdaa5c29f78628a"}
Jul 09 04:00:03.197965 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.197939    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bbe97f_cc76_4bb6_81f6_a8abbe5ada48.slice/crio-542f4b30da740ce62cbfa574ba491ebf76bb8611df35b400cd4f049f0dd8325f WatchSource:0}: Error finding container 542f4b30da740ce62cbfa574ba491ebf76bb8611df35b400cd4f049f0dd8325f: Status 404 returned error can't find the container with id 542f4b30da740ce62cbfa574ba491ebf76bb8611df35b400cd4f049f0dd8325f
Jul 09 04:00:03.214956 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.214932    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87958160_8026_4072_98a4_fe3b1faf6302.slice/crio-2a8dad50b70a9d0d6e97b488a0830b2329b3c253bab15c41b94ea9cdc88d3fb6 WatchSource:0}: Error finding container 2a8dad50b70a9d0d6e97b488a0830b2329b3c253bab15c41b94ea9cdc88d3fb6: Status 404 returned error can't find the container with id 2a8dad50b70a9d0d6e97b488a0830b2329b3c253bab15c41b94ea9cdc88d3fb6
Jul 09 04:00:03.305254 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.305224    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bad720_862e_46d4_a5e6_39b7fe607ff6.slice/crio-8f53df89de6591394c20d1f365d2cf43cea4162d1fefd0e93ec389ceb30d87ac WatchSource:0}: Error finding container 8f53df89de6591394c20d1f365d2cf43cea4162d1fefd0e93ec389ceb30d87ac: Status 404 returned error can't find the container with id 8f53df89de6591394c20d1f365d2cf43cea4162d1fefd0e93ec389ceb30d87ac
Jul 09 04:00:03.334252 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.334221    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5feb688e_54d8_4789_b86b_75f5b0a3b1b2.slice/crio-6788c7e445c503f77bbdd034b7360f3fc3fef8ce4911372d21a462e7e78e655e WatchSource:0}: Error finding container 6788c7e445c503f77bbdd034b7360f3fc3fef8ce4911372d21a462e7e78e655e: Status 404 returned error can't find the container with id 6788c7e445c503f77bbdd034b7360f3fc3fef8ce4911372d21a462e7e78e655e
Jul 09 04:00:03.388081 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.388055    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c97c43_619e_492d_8e51_7ec49f54eaea.slice/crio-a7a1a93fa6114774092df5a0d777de6fb6a7774954e815796d01d52080b8df2b WatchSource:0}: Error finding container a7a1a93fa6114774092df5a0d777de6fb6a7774954e815796d01d52080b8df2b: Status 404 returned error can't find the container with id a7a1a93fa6114774092df5a0d777de6fb6a7774954e815796d01d52080b8df2b
Jul 09 04:00:03.598039 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:03.598002    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f3f2d7_72e6_4467_9bbb_a18b0a99aa1f.slice/crio-6bb116e573612816f24a7317f16232d31327d65ec97fad7b175434ef1ef5bc2f WatchSource:0}: Error finding container 6bb116e573612816f24a7317f16232d31327d65ec97fad7b175434ef1ef5bc2f: Status 404 returned error can't find the container with id 6bb116e573612816f24a7317f16232d31327d65ec97fad7b175434ef1ef5bc2f
Jul 09 04:00:03.648901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.648827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:03.649093 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.649016    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:03.649160 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.649102    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:05.649080815 +0000 UTC m=+4.923458705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:03.749755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.749713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:03.749955 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:03.749820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:03.750021 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.749970    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:03.750021 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.749989    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:03.750021 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.750003    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:03.750179 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.750063    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:05.750044883 +0000 UTC m=+5.024422787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:03.750492 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.750464    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:03.750596 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:03.750532    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:05.750502372 +0000 UTC m=+5.024880274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:04.071362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.071193    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:55:02 +0000 UTC" deadline="2028-03-05 20:39:50.273264618 +0000 UTC"
Jul 09 04:00:04.071362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.071256    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14536h39m46.202013465s"
Jul 09 04:00:04.174206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.173966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:04.174206 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:04.174086    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:04.174509 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.174482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:04.174615 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:04.174587    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:04.184614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.184568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xbljd" event={"ID":"29bad720-862e-46d4-a5e6-39b7fe607ff6","Type":"ContainerStarted","Data":"8f53df89de6591394c20d1f365d2cf43cea4162d1fefd0e93ec389ceb30d87ac"}
Jul 09 04:00:04.197919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.197877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4nmdx" event={"ID":"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f","Type":"ContainerStarted","Data":"6bb116e573612816f24a7317f16232d31327d65ec97fad7b175434ef1ef5bc2f"}
Jul 09 04:00:04.205581 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.205544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-24cd2" event={"ID":"87958160-8026-4072-98a4-fe3b1faf6302","Type":"ContainerStarted","Data":"2a8dad50b70a9d0d6e97b488a0830b2329b3c253bab15c41b94ea9cdc88d3fb6"}
Jul 09 04:00:04.215641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.215606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"542f4b30da740ce62cbfa574ba491ebf76bb8611df35b400cd4f049f0dd8325f"}
Jul 09 04:00:04.219774 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.219741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerStarted","Data":"a7a1a93fa6114774092df5a0d777de6fb6a7774954e815796d01d52080b8df2b"}
Jul 09 04:00:04.223581 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.223552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4" event={"ID":"5feb688e-54d8-4789-b86b-75f5b0a3b1b2","Type":"ContainerStarted","Data":"6788c7e445c503f77bbdd034b7360f3fc3fef8ce4911372d21a462e7e78e655e"}
Jul 09 04:00:04.341425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.340089    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:04.911657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:04.911625    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 04:00:05.178093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:05.177807    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:05.178564 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.178160    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:05.667892 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:05.667812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:05.668061 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.667992    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:05.668061 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.668059    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:09.668041699 +0000 UTC m=+8.942419600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:05.768403 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:05.768367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:05.768600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:05.768428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:05.768600 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.768579    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:05.768712 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.768640    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:09.76862216 +0000 UTC m=+9.043000061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:05.769129 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.769032    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:05.769129 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.769051    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:05.769129 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.769064    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:05.769129 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:05.769105    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:09.769091379 +0000 UTC m=+9.043469267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:06.174256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:06.174223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:06.174443 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:06.174362    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:06.174643 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:06.174559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:06.174735 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:06.174678    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:07.173941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:07.173907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:07.174405 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:07.174051    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:08.174264 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:08.174226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:08.174746 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:08.174374    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:08.174746 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:08.174595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:08.174746 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:08.174681    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:09.174311 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:09.174279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:09.174756 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.174413    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:09.699616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:09.699542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:09.699806 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.699714    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:09.699806 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.699795    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:17.699773784 +0000 UTC m=+16.974151689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:09.799965 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:09.799919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:09.800127 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:09.799978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:09.800127 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800115    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:09.800247 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800180    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:17.800159391 +0000 UTC m=+17.074537301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:09.800635 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800435    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:09.800635 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800463    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:09.800635 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800519    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:09.800635 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:09.800576    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:17.800560676 +0000 UTC m=+17.074938565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:10.173737 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:10.173661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:10.173737 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:10.173708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:10.173950 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:10.173788    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:10.173950 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:10.173900    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:11.174904 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:11.174171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:11.174904 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:11.174281    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:12.173451 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:12.173385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:12.173659 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:12.173560    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:12.173733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:12.173399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:12.173835 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:12.173806    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:13.174400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:13.174358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:13.174817 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:13.174516    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:14.173362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:14.173320    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:14.173577 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:14.173448    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:14.173577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:14.173503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:14.173702 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:14.173630    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:15.173598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:15.173568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:15.174093 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:15.173692    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:16.174069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:16.174033    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:16.174546 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:16.174033    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:16.174546 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:16.174151    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:16.174546 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:16.174253    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:17.174165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:17.174068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:17.174631 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.174214    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:17.762658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:17.762623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:17.762843 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.762782    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:17.762908 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.762862    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:00:33.762838058 +0000 UTC m=+33.037215947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:17.863271 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:17.863226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:17.863283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863397    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863417    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863443    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863456    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:17.863456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863460    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:33.863440581 +0000 UTC m=+33.137818469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:17.863737 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:17.863519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:33.863503491 +0000 UTC m=+33.137881392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:18.173958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:18.173879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:18.173958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:18.173908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:18.174156 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:18.174010    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:18.174156 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:18.174137    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:19.173631 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:19.173592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:19.173797 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:19.173734    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:20.173400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:20.173366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:20.173835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:20.173409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:20.173835 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:20.173500    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:20.173835 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:20.173571    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:21.174940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:21.174894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:21.175408 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:21.174999    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:22.173897 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:22.173873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:22.173987 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:22.173967    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:22.174044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:22.174029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:22.174139 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:22.174121    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:22.260094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:22.260069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g5vfb" event={"ID":"ca467583-1e69-4d74-8c16-e5e033f74b6e","Type":"ContainerStarted","Data":"9d1a28504d82d747468aee4896755de01b476cd3de794d2e6301a2abf782cfa8"}
Jul 09 04:00:23.174369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.174150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:23.174546 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:23.174453    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:23.265249 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.265210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fltq9" event={"ID":"de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a","Type":"ContainerStarted","Data":"ecd9630cf05a09addcce4f96f1399ff8403cc6354c00d528911f90d6a0982efe"}
Jul 09 04:00:23.266890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.266855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xmpxf" event={"ID":"c8d6e6e9-c491-40bf-b405-df699fc273b6","Type":"ContainerStarted","Data":"a61cd60e85c608437922da6e81183705936d105bb3126c8706b55c982232f977"}
Jul 09 04:00:23.268428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.268405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal" event={"ID":"ed53264ef20f88c821b678c86f72e5cd","Type":"ContainerStarted","Data":"5352188a06ff8fede6adabc1c3995b98c21d465f4d5ea21bc57200ef496c7bf0"}
Jul 09 04:00:23.270164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.269991    2577 generic.go:356] "Generic (PLEG): container finished" podID="c9c32fa5fb84f7336ef4d31ff1741989" containerID="4df6ab084d0a7c9c3648e1e76983f95cb9bf941692b845b99e3dc607176afb70" exitCode=0
Jul 09 04:00:23.270273 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.270036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal" event={"ID":"c9c32fa5fb84f7336ef4d31ff1741989","Type":"ContainerDied","Data":"4df6ab084d0a7c9c3648e1e76983f95cb9bf941692b845b99e3dc607176afb70"}
Jul 09 04:00:23.271943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.271918    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="d8c6646fdad96455bdf8b45952a3dfd45e99eae89da2a897f227ebb8b649b0ee" exitCode=0
Jul 09 04:00:23.272047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.272011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"d8c6646fdad96455bdf8b45952a3dfd45e99eae89da2a897f227ebb8b649b0ee"}
Jul 09 04:00:23.273629 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.273593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4" event={"ID":"5feb688e-54d8-4789-b86b-75f5b0a3b1b2","Type":"ContainerStarted","Data":"4fbce82439dbf03e802355b4d414017341f58b379725354bc919dca709276787"}
Jul 09 04:00:23.275096 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.275073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xbljd" event={"ID":"29bad720-862e-46d4-a5e6-39b7fe607ff6","Type":"ContainerStarted","Data":"d0edbf0ad1b5ac21c124829a610cf9c1cb63ffdf0475f0cc22a3235d81376502"}
Jul 09 04:00:23.276764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.276742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4nmdx" event={"ID":"98f3f2d7-72e6-4467-9bbb-a18b0a99aa1f","Type":"ContainerStarted","Data":"1e6240ef1e43dcae4c5fa46085df72cfa8c63a3fa3a9b73ea35f815162bc4816"}
Jul 09 04:00:23.279897 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.279877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"44c14a7bb6f11f1879b046ae543d76807c38279c29fe59927679d52f5caca488"}
Jul 09 04:00:23.279988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.279945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"e26525c4da2b7c0cf9f5ba14dae8a16292b4e439e9bd6a7c6f019eb89c3a5d09"}
Jul 09 04:00:23.279988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.279975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"aced97cac51b87b980c4797e7834a6c205e5343a67c82bd0b9479d43fd4b36e5"}
Jul 09 04:00:23.280065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.280008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"eba923e825248c89d23e4197c4fa8e7f78f9ad63601e7df6c17a5797533a7f4f"}
Jul 09 04:00:23.280065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.280039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"1f0a469a97c3fe4e3351a49daf7425dd63b7cef4450de3f448d3d398d048e012"}
Jul 09 04:00:23.280138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.280097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"44b85a4521a4b76ba9e2ccb7d4efbc74b4ed598f5c37954ade6efdea0175235b"}
Jul 09 04:00:23.282946 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.282901    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fltq9" podStartSLOduration=3.328979984 podStartE2EDuration="22.282885641s" totalImagesPullingTime="18.953905657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:23.282142085 +0000 UTC m=+22.556520020" watchObservedRunningTime="2026-07-09 04:00:23.282885641 +0000 UTC m=+22.557263551"
Jul 09 04:00:23.310486 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.310393    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xbljd" podStartSLOduration=3.574674162 podStartE2EDuration="22.310379361s" totalImagesPullingTime="18.735705199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:23.309985259 +0000 UTC m=+22.584363169" watchObservedRunningTime="2026-07-09 04:00:23.310379361 +0000 UTC m=+22.584757268"
Jul 09 04:00:23.322661 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.322618    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xmpxf" podStartSLOduration=3.371893986 podStartE2EDuration="22.322604924s" totalImagesPullingTime="18.950710938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:23.322280242 +0000 UTC m=+22.596658187" watchObservedRunningTime="2026-07-09 04:00:23.322604924 +0000 UTC m=+22.596982831"
Jul 09 04:00:23.342765 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.342717    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4nmdx" podStartSLOduration=3.823808047 podStartE2EDuration="22.342707491s" totalImagesPullingTime="18.518899444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:23.342499594 +0000 UTC m=+22.616877529" watchObservedRunningTime="2026-07-09 04:00:23.342707491 +0000 UTC m=+22.617085398"
Jul 09 04:00:23.359931 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:23.359887    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-255.ec2.internal" podStartSLOduration=21.359873981 podStartE2EDuration="21.359873981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:23.359461434 +0000 UTC m=+22.633839342" watchObservedRunningTime="2026-07-09 04:00:23.359873981 +0000 UTC m=+22.634251889"
Jul 09 04:00:24.173567 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.173521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:24.173862 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:24.173656    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:24.173862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.173715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:24.173862 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:24.173811    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:24.283315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.283272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal" event={"ID":"c9c32fa5fb84f7336ef4d31ff1741989","Type":"ContainerStarted","Data":"7f0836c6aedf118d839e2065ccbc5c13f42f90faeefdb1e5a5f33da83b2f6df2"}
Jul 09 04:00:24.284609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.284584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g5vfb" event={"ID":"ca467583-1e69-4d74-8c16-e5e033f74b6e","Type":"ContainerStarted","Data":"f42451d3757b3a753a9903329639611ea12711a057fbc3f549cbdc78ea3a5a8d"}
Jul 09 04:00:24.285789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.285763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-24cd2" event={"ID":"87958160-8026-4072-98a4-fe3b1faf6302","Type":"ContainerStarted","Data":"c104107fd97de48d3c4c4f9e72080ce5645ed309ca5614a8afe01726924774ae"}
Jul 09 04:00:24.313333 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.313289    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-255.ec2.internal" podStartSLOduration=22.313275186 podStartE2EDuration="22.313275186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:24.297308134 +0000 UTC m=+23.571686039" watchObservedRunningTime="2026-07-09 04:00:24.313275186 +0000 UTC m=+23.587653092"
Jul 09 04:00:24.326350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.326307    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g5vfb" podStartSLOduration=21.714038428 podStartE2EDuration="23.326294271s" totalImagesPullingTime="1.612255843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:24.325971024 +0000 UTC m=+23.600348932" watchObservedRunningTime="2026-07-09 04:00:24.326294271 +0000 UTC m=+23.600672179"
Jul 09 04:00:24.326605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.326577    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-24cd2" podStartSLOduration=4.500471427 podStartE2EDuration="23.326570529s" totalImagesPullingTime="18.826099102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:24.313243446 +0000 UTC m=+23.587621353" watchObservedRunningTime="2026-07-09 04:00:24.326570529 +0000 UTC m=+23.600948415"
Jul 09 04:00:24.581047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.581020    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:24.581664 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.581644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:24.640916 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:24.640893    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 04:00:25.106730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.106637    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T04:00:24.640910952Z","UUID":"f174fd52-6e55-44ec-a160-b4252a80783c","Handler":null,"Name":"","Endpoint":""}
Jul 09 04:00:25.108405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.108382    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 04:00:25.108550 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.108429    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 04:00:25.174251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.174226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:25.174398 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:25.174348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:25.290411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.290374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4" event={"ID":"5feb688e-54d8-4789-b86b-75f5b0a3b1b2","Type":"ContainerStarted","Data":"6778a59d7091b78585939682841621de98702d3724fe65c525748e8d841d31a4"}
Jul 09 04:00:25.294249 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.294174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"9b4334d51663111ecd9549114e2d1dac612953c7a283ae00608f62ee59ee9bc3"}
Jul 09 04:00:25.294791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.294684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:25.295020 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:25.295002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g5vfb"
Jul 09 04:00:26.173979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:26.173938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:26.173979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:26.173968    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:26.174257 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:26.174084    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:26.174257 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:26.174222    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:26.299366 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:26.299332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4" event={"ID":"5feb688e-54d8-4789-b86b-75f5b0a3b1b2","Type":"ContainerStarted","Data":"a1c2bb10c391a74aadaed40180c402c7b96c7e89509bede4b19384327e8c7a6f"}
Jul 09 04:00:26.316118 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:26.316069    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt4s4" podStartSLOduration=3.8341963359999998 podStartE2EDuration="25.316056061s" totalImagesPullingTime="21.481859725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:26.315439062 +0000 UTC m=+25.589816970" watchObservedRunningTime="2026-07-09 04:00:26.316056061 +0000 UTC m=+25.590433969"
Jul 09 04:00:27.173785 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:27.173752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:27.173957 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:27.173859    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:28.174370 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.174250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:28.174716 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.174250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:28.174716 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:28.174447    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:28.174716 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:28.174561    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:28.306614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.306575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" event={"ID":"83bbe97f-cc76-4bb6-81f6-a8abbe5ada48","Type":"ContainerStarted","Data":"539e692fbd07fa63bc59ff9d52987d976b4face974a3b1c02c7adf765f150645"}
Jul 09 04:00:28.306915 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.306898    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:28.307002 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.306955    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:28.321607 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.321585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:28.321741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.321714    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:28.332237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:28.332197    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt" podStartSLOduration=8.137696393 podStartE2EDuration="27.332183832s" totalImagesPullingTime="19.194487439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:28.331526276 +0000 UTC m=+27.605904202" watchObservedRunningTime="2026-07-09 04:00:28.332183832 +0000 UTC m=+27.606561740"
Jul 09 04:00:29.174160 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:29.174092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:29.174342 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:29.174213    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:29.310088 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:29.310051    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="7718f26473f8fd1359a680611ed481e8d750dbcba5306d2c7336a4b2de2aa332" exitCode=0
Jul 09 04:00:29.310814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:29.310144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"7718f26473f8fd1359a680611ed481e8d750dbcba5306d2c7336a4b2de2aa332"}
Jul 09 04:00:29.310814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:29.310190    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 04:00:29.523776 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:29.523706    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:00:30.123339 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.123167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9cr6q"]
Jul 09 04:00:30.123541 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.123400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:30.123541 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:30.123492    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:30.125984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.125961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-986sz"]
Jul 09 04:00:30.126092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.126057    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:30.126172 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:30.126153    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:30.126498 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.126463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-st6ww"]
Jul 09 04:00:30.126603 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:30.126591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:30.126683 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:30.126666    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:32.173889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.173850    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:32.174256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.173951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:32.174256 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:32.173957    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:32.174256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.173980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:32.174256 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:32.174062    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:32.174256 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:32.174127    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:32.286982 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.286942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:32.318784 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.318752    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="55353f297f8b97239e13602f10ca4002409045b12f760df2389b16458f998ebe" exitCode=0
Jul 09 04:00:32.318953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.318790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"55353f297f8b97239e13602f10ca4002409045b12f760df2389b16458f998ebe"}
Jul 09 04:00:32.375879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:32.375842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:33.789236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:33.789206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:33.789586 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.789321    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:33.789586 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.789371    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:01:05.789358336 +0000 UTC m=+65.063736222 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 04:00:33.889879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:33.889850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:33.890003 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:33.889915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:33.890003 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.889984    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:33.890078 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.890007    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 04:00:33.890078 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.890026    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 04:00:33.890078 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.890029    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret podName:6dcb5037-c133-44cf-9617-ec0a07a55301 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:05.890017978 +0000 UTC m=+65.164395867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret") pod "global-pull-secret-syncer-9cr6q" (UID: "6dcb5037-c133-44cf-9617-ec0a07a55301") : object "kube-system"/"original-pull-secret" not registered
Jul 09 04:00:33.890078 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.890036    2577 projected.go:196] Error preparing data for projected volume kube-api-access-nlqnr for pod openshift-network-diagnostics/network-check-target-986sz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:33.890078 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:33.890075    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr podName:0ab04f9a-0f9b-459a-9eee-ea0eca7b6562 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:05.890062718 +0000 UTC m=+65.164440617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nlqnr" (UniqueName: "kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr") pod "network-check-target-986sz" (UID: "0ab04f9a-0f9b-459a-9eee-ea0eca7b6562") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 04:00:34.174347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.174272    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:34.174347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.174285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:34.174550 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:34.174363    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cr6q" podUID="6dcb5037-c133-44cf-9617-ec0a07a55301"
Jul 09 04:00:34.174550 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.174389    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:34.174550 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:34.174497    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-st6ww" podUID="ba58ffef-ea23-486c-ad2e-be920dc8173c"
Jul 09 04:00:34.174654 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:34.174571    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-986sz" podUID="0ab04f9a-0f9b-459a-9eee-ea0eca7b6562"
Jul 09 04:00:34.322135 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.322102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:34.327856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.327824    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="7c848b9b1bb2076803133a1a7110fa84c1ef1999c6fcd21d8e7171d96a0ba593" exitCode=0
Jul 09 04:00:34.327970 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:34.327866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"7c848b9b1bb2076803133a1a7110fa84c1ef1999c6fcd21d8e7171d96a0ba593"}
Jul 09 04:00:35.069841 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.069599    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-255.ec2.internal" event="NodeReady"
Jul 09 04:00:35.070254 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.069922    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 04:00:35.113706 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.113675    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wwrtx"]
Jul 09 04:00:35.116696 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.116681    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bf7hv"]
Jul 09 04:00:35.116817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.116799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.118676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.118656    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 04:00:35.118786 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.118721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 04:00:35.118830 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.118786    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wgmf2\""
Jul 09 04:00:35.120196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.120175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.121673 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.121649    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 04:00:35.122011 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.121987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 04:00:35.122134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.122036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dfx4h\""
Jul 09 04:00:35.122134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.122058    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 04:00:35.129344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.129321    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bf7hv"]
Jul 09 04:00:35.129429 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.129378    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wwrtx"]
Jul 09 04:00:35.199648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eff9c62-944c-454a-8dd3-830e3fcc7c23-cert\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.199828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4538ad5-df7b-497a-b315-8b49ced9d705-config-volume\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.199828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgg6q\" (UniqueName: \"kubernetes.io/projected/4eff9c62-944c-454a-8dd3-830e3fcc7c23-kube-api-access-hgg6q\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.199828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.199828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4538ad5-df7b-497a-b315-8b49ced9d705-tmp-dir\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.200004 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.199848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pml84\" (UniqueName: \"kubernetes.io/projected/c4538ad5-df7b-497a-b315-8b49ced9d705-kube-api-access-pml84\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eff9c62-944c-454a-8dd3-830e3fcc7c23-cert\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.301050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4538ad5-df7b-497a-b315-8b49ced9d705-config-volume\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgg6q\" (UniqueName: \"kubernetes.io/projected/4eff9c62-944c-454a-8dd3-830e3fcc7c23-kube-api-access-hgg6q\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.301283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4538ad5-df7b-497a-b315-8b49ced9d705-tmp-dir\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pml84\" (UniqueName: \"kubernetes.io/projected/c4538ad5-df7b-497a-b315-8b49ced9d705-kube-api-access-pml84\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301516 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:35.301488    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:35.301644 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:35.301564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:35.80154641 +0000 UTC m=+35.075924296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:35.301768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301740    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4538ad5-df7b-497a-b315-8b49ced9d705-tmp-dir\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.301875 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.301748    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4538ad5-df7b-497a-b315-8b49ced9d705-config-volume\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.305939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.305918    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eff9c62-944c-454a-8dd3-830e3fcc7c23-cert\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.310167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.310124    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pml84\" (UniqueName: \"kubernetes.io/projected/c4538ad5-df7b-497a-b315-8b49ced9d705-kube-api-access-pml84\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.310331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.310293    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgg6q\" (UniqueName: \"kubernetes.io/projected/4eff9c62-944c-454a-8dd3-830e3fcc7c23-kube-api-access-hgg6q\") pod \"ingress-canary-bf7hv\" (UID: \"4eff9c62-944c-454a-8dd3-830e3fcc7c23\") " pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.435911 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.435873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bf7hv"
Jul 09 04:00:35.568740 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.568707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:35.593535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.593480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bf7hv"]
Jul 09 04:00:35.598731 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:00:35.598691    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eff9c62_944c_454a_8dd3_830e3fcc7c23.slice/crio-7de97505feb2777da9c0f3b13be99efe16bb7596f1ece80972cd6f38795c69c1 WatchSource:0}: Error finding container 7de97505feb2777da9c0f3b13be99efe16bb7596f1ece80972cd6f38795c69c1: Status 404 returned error can't find the container with id 7de97505feb2777da9c0f3b13be99efe16bb7596f1ece80972cd6f38795c69c1
Jul 09 04:00:35.667630 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.667597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:35.804128 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:35.804049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:35.804266 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:35.804173    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:35.804266 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:35.804242    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:36.804221016 +0000 UTC m=+36.078598907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:36.174515 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.173835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:00:36.174515 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.173883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:00:36.174515 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.174063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:00:36.176358 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.176138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 04:00:36.176358 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.176164    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 04:00:36.176592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.176403    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 04:00:36.176681 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.176610    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fn82r\""
Jul 09 04:00:36.177307 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.177287    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-25smd\""
Jul 09 04:00:36.177714 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.177389    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 04:00:36.334886 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.334856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bf7hv" event={"ID":"4eff9c62-944c-454a-8dd3-830e3fcc7c23","Type":"ContainerStarted","Data":"7de97505feb2777da9c0f3b13be99efe16bb7596f1ece80972cd6f38795c69c1"}
Jul 09 04:00:36.813664 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:36.813622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:36.813839 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:36.813746    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:36.813918 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:36.813844    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:38.813827249 +0000 UTC m=+38.088205148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:37.032883 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:37.032837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:37.107422 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:37.107387    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:38.829957 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:38.829751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:38.830412 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:38.829917    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:38.830412 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:38.830039    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:42.830021564 +0000 UTC m=+42.104399452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:39.305712 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:39.305679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:40.346535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:40.346422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bf7hv" event={"ID":"4eff9c62-944c-454a-8dd3-830e3fcc7c23","Type":"ContainerStarted","Data":"be24e823db83718bdcf3b0a961f7b207105d47ebce2db9dcfe1c26d9aa67a88d"}
Jul 09 04:00:40.349091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:40.349069    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="98f94c96027794a51670ba1404a0e913ed4a08b6376701714d413aeecb76b0f4" exitCode=0
Jul 09 04:00:40.349224 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:40.349119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"98f94c96027794a51670ba1404a0e913ed4a08b6376701714d413aeecb76b0f4"}
Jul 09 04:00:40.360646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:40.360604    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bf7hv" podStartSLOduration=1.251423579 podStartE2EDuration="5.360588159s" totalImagesPullingTime="4.10916458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:40.359991592 +0000 UTC m=+39.634369525" watchObservedRunningTime="2026-07-09 04:00:40.360588159 +0000 UTC m=+39.634966045"
Jul 09 04:00:41.354196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:41.354158    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1c97c43-619e-492d-8e51-7ec49f54eaea" containerID="a5b46aa210912fac000dde27e81cf7dd7da500237fbbd5a4b641a24ab143eb69" exitCode=0
Jul 09 04:00:41.354583 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:41.354241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerDied","Data":"a5b46aa210912fac000dde27e81cf7dd7da500237fbbd5a4b641a24ab143eb69"}
Jul 09 04:00:42.359131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:42.359101    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-88f9s" event={"ID":"c1c97c43-619e-492d-8e51-7ec49f54eaea","Type":"ContainerStarted","Data":"7ffb478cb0b61210e6ede875b53ad375bc13b57d0a5d928071b485ef59f4b30f"}
Jul 09 04:00:42.379412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:42.379363    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-88f9s" podStartSLOduration=8.474554434 podStartE2EDuration="41.37935037s" totalImagesPullingTime="32.904795936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:42.378673781 +0000 UTC m=+41.653051707" watchObservedRunningTime="2026-07-09 04:00:42.37935037 +0000 UTC m=+41.653728278"
Jul 09 04:00:42.857543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:42.857514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:42.857676 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:42.857618    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:42.857676 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:42.857667    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:50.857653956 +0000 UTC m=+50.132031842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:43.063868 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:43.063832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:43.128057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:43.127990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:45.888598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:45.888546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:50.911999 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:50.911963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:00:50.912436 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:50.912111    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:00:50.912436 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:00:50.912188    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:06.912170342 +0000 UTC m=+66.186548248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:00:52.989952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:52.989912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:53.056865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:53.056837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:55.582835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:55.582803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:55.682743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:00:55.682709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:01.331982 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:01.331956    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jchmt"
Jul 09 04:01:01.448122 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:01.448086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:01.503064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:01.503033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:05.816734 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.816696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:01:05.818818 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.818799    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 04:01:05.827083 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:05.827064    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 04:01:05.827143 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:05.827119    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs podName:ba58ffef-ea23-486c-ad2e-be920dc8173c nodeName:}" failed. No retries permitted until 2026-07-09 04:02:09.82710491 +0000 UTC m=+129.101482796 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs") pod "network-metrics-daemon-st6ww" (UID: "ba58ffef-ea23-486c-ad2e-be920dc8173c") : secret "metrics-daemon-secret" not found
Jul 09 04:01:05.917984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.917950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:01:05.918110 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.917991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:01:05.920236 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.920219    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 04:01:05.920345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.920331    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 04:01:05.930068 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.930048    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 04:01:05.931529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.931498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dcb5037-c133-44cf-9617-ec0a07a55301-original-pull-secret\") pod \"global-pull-secret-syncer-9cr6q\" (UID: \"6dcb5037-c133-44cf-9617-ec0a07a55301\") " pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:01:05.941363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:05.941340    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlqnr\" (UniqueName: \"kubernetes.io/projected/0ab04f9a-0f9b-459a-9eee-ea0eca7b6562-kube-api-access-nlqnr\") pod \"network-check-target-986sz\" (UID: \"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562\") " pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:01:06.189920 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.189890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cr6q"
Jul 09 04:01:06.205793 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.205774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-25smd\""
Jul 09 04:01:06.214179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.214158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:01:06.319901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.319869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9cr6q"]
Jul 09 04:01:06.324047 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:06.324023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcb5037_c133_44cf_9617_ec0a07a55301.slice/crio-5319c605a99e31c1b294daee7a3aebbf5ce7bd48ed6e92738261b657cc1ff1c5 WatchSource:0}: Error finding container 5319c605a99e31c1b294daee7a3aebbf5ce7bd48ed6e92738261b657cc1ff1c5: Status 404 returned error can't find the container with id 5319c605a99e31c1b294daee7a3aebbf5ce7bd48ed6e92738261b657cc1ff1c5
Jul 09 04:01:06.337085 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.337063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-986sz"]
Jul 09 04:01:06.340039 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:06.340021    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab04f9a_0f9b_459a_9eee_ea0eca7b6562.slice/crio-a95d5aa8f01e9c68f20e203bbfd0f687d32da3c76f88be0c739f18a8dab0da32 WatchSource:0}: Error finding container a95d5aa8f01e9c68f20e203bbfd0f687d32da3c76f88be0c739f18a8dab0da32: Status 404 returned error can't find the container with id a95d5aa8f01e9c68f20e203bbfd0f687d32da3c76f88be0c739f18a8dab0da32
Jul 09 04:01:06.411350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.411307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-986sz" event={"ID":"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562","Type":"ContainerStarted","Data":"a95d5aa8f01e9c68f20e203bbfd0f687d32da3c76f88be0c739f18a8dab0da32"}
Jul 09 04:01:06.412313 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.412282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9cr6q" event={"ID":"6dcb5037-c133-44cf-9617-ec0a07a55301","Type":"ContainerStarted","Data":"5319c605a99e31c1b294daee7a3aebbf5ce7bd48ed6e92738261b657cc1ff1c5"}
Jul 09 04:01:06.926010 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:06.925971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:06.926509 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:06.926104    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 04:01:06.926509 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:06.926163    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls podName:c4538ad5-df7b-497a-b315-8b49ced9d705 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:38.926147625 +0000 UTC m=+98.200525511 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls") pod "dns-default-wwrtx" (UID: "c4538ad5-df7b-497a-b315-8b49ced9d705") : secret "dns-default-metrics-tls" not found
Jul 09 04:01:07.202015 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:07.201919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:07.256522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:07.256479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:10.427457 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:10.427390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-986sz" event={"ID":"0ab04f9a-0f9b-459a-9eee-ea0eca7b6562","Type":"ContainerStarted","Data":"064d359295c7e05ca0e847ccd3dc48ba608ad6c78ccaf8b11a1d7797e5cbad41"}
Jul 09 04:01:10.427935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:10.427607    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:01:10.446935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:10.446871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-986sz" podStartSLOduration=66.435810181 podStartE2EDuration="1m9.446854771s" totalImagesPullingTime="3.01104459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:10.446309239 +0000 UTC m=+69.720687149" watchObservedRunningTime="2026-07-09 04:01:10.446854771 +0000 UTC m=+69.721232679"
Jul 09 04:01:11.432155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:11.432119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9cr6q" event={"ID":"6dcb5037-c133-44cf-9617-ec0a07a55301","Type":"ContainerStarted","Data":"b3c1b10af07e61d7044cd0e8e0582f24f47c705afb4a2faae187ff1dccda6460"}
Jul 09 04:01:11.448656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:11.448597    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9cr6q" podStartSLOduration=65.475522107 podStartE2EDuration="1m10.448577666s" totalImagesPullingTime="4.973055559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:11.446950153 +0000 UTC m=+70.721328084" watchObservedRunningTime="2026-07-09 04:01:11.448577666 +0000 UTC m=+70.722955575"
Jul 09 04:01:18.614032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.613995    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"]
Jul 09 04:01:18.616457 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.616442    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"]
Jul 09 04:01:18.616584 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.616566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.618680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.618660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.619056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.619036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 04:01:18.619204 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.619070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8vpgd\""
Jul 09 04:01:18.619204 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.619078    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 04:01:18.619873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.619851    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 04:01:18.620361 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.620332    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 09 04:01:18.620495 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.620459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 09 04:01:18.620795 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.620779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dc6tx\""
Jul 09 04:01:18.620862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.620803    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 09 04:01:18.621843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.621828    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 09 04:01:18.632927 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.632907    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"]
Jul 09 04:01:18.633648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.633628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"]
Jul 09 04:01:18.703383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.703352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/304065f4-8a85-4032-87cc-6eedf9065811-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.703383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.703384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.703617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.703410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24f4p\" (UniqueName: \"kubernetes.io/projected/304065f4-8a85-4032-87cc-6eedf9065811-kube-api-access-24f4p\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.703617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.703500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw5v7\" (UniqueName: \"kubernetes.io/projected/9c041576-2b96-42f5-a4eb-2744f06d267f-kube-api-access-jw5v7\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.703617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.703559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/304065f4-8a85-4032-87cc-6eedf9065811-config\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.717425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.717394    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"]
Jul 09 04:01:18.719602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.719587    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2vxxm"]
Jul 09 04:01:18.719728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.719712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.721579 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.721527    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ln6lg\""
Jul 09 04:01:18.721720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.721696    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 04:01:18.721720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.721696    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 04:01:18.721816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.721765    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.725033 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.725014    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 09 04:01:18.725378 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.725352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 09 04:01:18.725463 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.725408    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 09 04:01:18.725463 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.725417    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 09 04:01:18.726042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.726003    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xt68s\""
Jul 09 04:01:18.732529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.732509    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 09 04:01:18.740764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.740743    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2vxxm"]
Jul 09 04:01:18.741480 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.741443    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"]
Jul 09 04:01:18.775993 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.775960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:18.803862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/304065f4-8a85-4032-87cc-6eedf9065811-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24f4p\" (UniqueName: \"kubernetes.io/projected/304065f4-8a85-4032-87cc-6eedf9065811-kube-api-access-24f4p\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw5v7\" (UniqueName: \"kubernetes.io/projected/9c041576-2b96-42f5-a4eb-2744f06d267f-kube-api-access-jw5v7\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-config\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.803997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/304065f4-8a85-4032-87cc-6eedf9065811-config\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.804019 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:18.804000    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 04:01:18.804363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.804025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7de353e-7173-43ab-879e-97b27640cbc6-serving-cert\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.804363 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:18.804068    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls podName:9c041576-2b96-42f5-a4eb-2744f06d267f nodeName:}" failed. No retries permitted until 2026-07-09 04:01:19.304047258 +0000 UTC m=+78.578425145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-scnwx" (UID: "9c041576-2b96-42f5-a4eb-2744f06d267f") : secret "samples-operator-tls" not found
Jul 09 04:01:18.804363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.804133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk4mk\" (UniqueName: \"kubernetes.io/projected/d7de353e-7173-43ab-879e-97b27640cbc6-kube-api-access-jk4mk\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.804363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.804237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04793198-4033-4011-bebf-d70b4b2b8185-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.804363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.804271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-trusted-ca\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.804667 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.804649    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/304065f4-8a85-4032-87cc-6eedf9065811-config\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.806141 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.806124    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/304065f4-8a85-4032-87cc-6eedf9065811-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.811744 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.811722    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw5v7\" (UniqueName: \"kubernetes.io/projected/9c041576-2b96-42f5-a4eb-2744f06d267f-kube-api-access-jw5v7\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:18.811845 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.811791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24f4p\" (UniqueName: \"kubernetes.io/projected/304065f4-8a85-4032-87cc-6eedf9065811-kube-api-access-24f4p\") pod \"service-ca-operator-5d46d9ffd4-txlvp\" (UID: \"304065f4-8a85-4032-87cc-6eedf9065811\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.875405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.875316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:18.904827 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7de353e-7173-43ab-879e-97b27640cbc6-serving-cert\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.904932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk4mk\" (UniqueName: \"kubernetes.io/projected/d7de353e-7173-43ab-879e-97b27640cbc6-kube-api-access-jk4mk\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.904932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04793198-4033-4011-bebf-d70b4b2b8185-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.904932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-trusted-ca\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.904932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.905116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.904945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-config\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.905116 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:18.905030    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:18.905116 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:18.905086    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:19.405069325 +0000 UTC m=+78.679447218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:18.905607 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.905587    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04793198-4033-4011-bebf-d70b4b2b8185-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:18.905704 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.905677    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-config\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.905754 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.905741    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7de353e-7173-43ab-879e-97b27640cbc6-trusted-ca\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.907213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.907194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7de353e-7173-43ab-879e-97b27640cbc6-serving-cert\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.913266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.913249    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk4mk\" (UniqueName: \"kubernetes.io/projected/d7de353e-7173-43ab-879e-97b27640cbc6-kube-api-access-jk4mk\") pod \"console-operator-68b757fb67-2vxxm\" (UID: \"d7de353e-7173-43ab-879e-97b27640cbc6\") " pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:18.931355 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.931336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"
Jul 09 04:01:18.993866 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.993817    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:01:18.996569 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.996519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:18.998792 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.998760    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 04:01:18.998905 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.998791    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 04:01:18.998905 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.998807    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 04:01:18.999076 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:18.999061    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brrqc\""
Jul 09 04:01:19.005978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.004766    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 04:01:19.020685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.020621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:01:19.034606 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.034575    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:19.056128 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.056102    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp"]
Jul 09 04:01:19.058424 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:19.058400    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304065f4_8a85_4032_87cc_6eedf9065811.slice/crio-b5b1d0015246350bd6fb2ccc7bdeba10ec8d61918a706718b6bcb95c227bf93a WatchSource:0}: Error finding container b5b1d0015246350bd6fb2ccc7bdeba10ec8d61918a706718b6bcb95c227bf93a: Status 404 returned error can't find the container with id b5b1d0015246350bd6fb2ccc7bdeba10ec8d61918a706718b6bcb95c227bf93a
Jul 09 04:01:19.106940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.106908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107080 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.106993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107080 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107080 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf5k7\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.107179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.107151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.152699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.152625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2vxxm"]
Jul 09 04:01:19.156062 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:19.156034    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7de353e_7173_43ab_879e_97b27640cbc6.slice/crio-cc7422d6386f8c3ad4c312f1cf9ade7d7556bdc7df978b98b29bfca1f46c5b64 WatchSource:0}: Error finding container cc7422d6386f8c3ad4c312f1cf9ade7d7556bdc7df978b98b29bfca1f46c5b64: Status 404 returned error can't find the container with id cc7422d6386f8c3ad4c312f1cf9ade7d7556bdc7df978b98b29bfca1f46c5b64
Jul 09 04:01:19.208252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208526 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208526 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5k7\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208624 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208624 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.208571    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 04:01:19.208624 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.208590    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6877dd5876-glhmh: secret "image-registry-tls" not found
Jul 09 04:01:19.208755 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.208647    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls podName:66a3b7c6-3e76-4ba0-9998-0c765ed326fc nodeName:}" failed. No retries permitted until 2026-07-09 04:01:19.708627388 +0000 UTC m=+78.983005295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls") pod "image-registry-6877dd5876-glhmh" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc") : secret "image-registry-tls" not found
Jul 09 04:01:19.208755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208740    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208861 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.208966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.208934    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.209678 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.209660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.210921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.210904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.210965 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.210941    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.216256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.216229    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.216360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.216346    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf5k7\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.309707 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.309689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:19.309808 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.309796    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 04:01:19.309852 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.309844    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls podName:9c041576-2b96-42f5-a4eb-2744f06d267f nodeName:}" failed. No retries permitted until 2026-07-09 04:01:20.309832056 +0000 UTC m=+79.584209942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-scnwx" (UID: "9c041576-2b96-42f5-a4eb-2744f06d267f") : secret "samples-operator-tls" not found
Jul 09 04:01:19.410803 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.410743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:19.410895 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.410848    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:19.410957 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.410906    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:20.410889987 +0000 UTC m=+79.685267873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:19.452446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.452417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" event={"ID":"d7de353e-7173-43ab-879e-97b27640cbc6","Type":"ContainerStarted","Data":"cc7422d6386f8c3ad4c312f1cf9ade7d7556bdc7df978b98b29bfca1f46c5b64"}
Jul 09 04:01:19.453398 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.453379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp" event={"ID":"304065f4-8a85-4032-87cc-6eedf9065811","Type":"ContainerStarted","Data":"b5b1d0015246350bd6fb2ccc7bdeba10ec8d61918a706718b6bcb95c227bf93a"}
Jul 09 04:01:19.712642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:19.712573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:19.713044 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.712700    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 04:01:19.713044 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.712715    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6877dd5876-glhmh: secret "image-registry-tls" not found
Jul 09 04:01:19.713044 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:19.712777    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls podName:66a3b7c6-3e76-4ba0-9998-0c765ed326fc nodeName:}" failed. No retries permitted until 2026-07-09 04:01:20.712760999 +0000 UTC m=+79.987138888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls") pod "image-registry-6877dd5876-glhmh" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc") : secret "image-registry-tls" not found
Jul 09 04:01:20.317943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:20.317909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:20.318123 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.318069    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 04:01:20.318170 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.318143    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls podName:9c041576-2b96-42f5-a4eb-2744f06d267f nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.318120189 +0000 UTC m=+81.592498094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-scnwx" (UID: "9c041576-2b96-42f5-a4eb-2744f06d267f") : secret "samples-operator-tls" not found
Jul 09 04:01:20.418413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:20.418373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:20.418566 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.418539    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:20.418622 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.418614    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.4185814 +0000 UTC m=+81.692959305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:20.720743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:20.720711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:20.721132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.720887    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 04:01:20.721132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.720910    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6877dd5876-glhmh: secret "image-registry-tls" not found
Jul 09 04:01:20.721132 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:20.720981    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls podName:66a3b7c6-3e76-4ba0-9998-0c765ed326fc nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.720958908 +0000 UTC m=+81.995336799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls") pod "image-registry-6877dd5876-glhmh" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc") : secret "image-registry-tls" not found
Jul 09 04:01:21.084562 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.084413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:21.143304 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.143200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:21.949685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.949651    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vlt7p"]
Jul 09 04:01:21.952926 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.952904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:21.955144 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.955126    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 04:01:21.955370 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.955356    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 04:01:21.955676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.955659    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 04:01:21.955764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.955663    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 04:01:21.956107 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.956097    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9vrhl\""
Jul 09 04:01:21.973300 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.973249    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlt7p"]
Jul 09 04:01:21.976171 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:21.976140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:22.031502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.031463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkctr\" (UniqueName: \"kubernetes.io/projected/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-api-access-kkctr\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.031620 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.031547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-crio-socket\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.031620 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.031567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-data-volume\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.031620 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.031586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.031767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.031720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.052862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.052838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:22.132438 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-crio-socket\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-data-volume\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-crio-socket\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132766 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132766 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.132693    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:22.132766 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkctr\" (UniqueName: \"kubernetes.io/projected/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-api-access-kkctr\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132766 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.132744    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.632726439 +0000 UTC m=+81.907104326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:22.132941 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.132776    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 04:01:22.132941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.132796    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-data-volume\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.132941 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.132844    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.632832064 +0000 UTC m=+81.907209954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : configmap "kube-rbac-proxy" not found
Jul 09 04:01:22.142239 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.142220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkctr\" (UniqueName: \"kubernetes.io/projected/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-api-access-kkctr\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.335149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.335061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:22.335296 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.335197    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 04:01:22.335296 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.335266    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls podName:9c041576-2b96-42f5-a4eb-2744f06d267f nodeName:}" failed. No retries permitted until 2026-07-09 04:01:26.335248969 +0000 UTC m=+85.609626874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-scnwx" (UID: "9c041576-2b96-42f5-a4eb-2744f06d267f") : secret "samples-operator-tls" not found
Jul 09 04:01:22.435427 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.435395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:22.435585 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.435511    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:22.435585 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.435566    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:26.435550493 +0000 UTC m=+85.709928379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:22.452190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.452160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:22.462152 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.462134    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/0.log"
Jul 09 04:01:22.462243 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.462170    2577 generic.go:356] "Generic (PLEG): container finished" podID="d7de353e-7173-43ab-879e-97b27640cbc6" containerID="4bae03f81189533bb54b7339acc3fd04355712d23336e23425c4582185eceb2e" exitCode=255
Jul 09 04:01:22.462294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.462258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" event={"ID":"d7de353e-7173-43ab-879e-97b27640cbc6","Type":"ContainerDied","Data":"4bae03f81189533bb54b7339acc3fd04355712d23336e23425c4582185eceb2e"}
Jul 09 04:01:22.462481 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.462449    2577 scope.go:117] "RemoveContainer" containerID="4bae03f81189533bb54b7339acc3fd04355712d23336e23425c4582185eceb2e"
Jul 09 04:01:22.463739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.463718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp" event={"ID":"304065f4-8a85-4032-87cc-6eedf9065811","Type":"ContainerStarted","Data":"ada533f880e1a5fbaf40226fc68eeda639575ec3b4f95505d504782980e04cae"}
Jul 09 04:01:22.500089 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.500036    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp" podStartSLOduration=1.84529801 podStartE2EDuration="4.50001869s" totalImagesPullingTime="2.65472068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:22.499982872 +0000 UTC m=+81.774360804" watchObservedRunningTime="2026-07-09 04:01:22.50001869 +0000 UTC m=+81.774396601"
Jul 09 04:01:22.529513 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.529483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:22.636586 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.636561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.636720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.636612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:22.636720 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.636710    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 04:01:22.636825 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.636714    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:22.636825 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.636758    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:23.636745745 +0000 UTC m=+82.911123630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : configmap "kube-rbac-proxy" not found
Jul 09 04:01:22.636825 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.636770    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:23.636763974 +0000 UTC m=+82.911141859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:22.737053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:22.737027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:22.740723 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.737550    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 04:01:22.740723 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.737572    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6877dd5876-glhmh: secret "image-registry-tls" not found
Jul 09 04:01:22.740723 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:22.737639    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls podName:66a3b7c6-3e76-4ba0-9998-0c765ed326fc nodeName:}" failed. No retries permitted until 2026-07-09 04:01:26.737621197 +0000 UTC m=+86.011999098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls") pod "image-registry-6877dd5876-glhmh" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc") : secret "image-registry-tls" not found
Jul 09 04:01:23.468812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.468780    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/1.log"
Jul 09 04:01:23.469246 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.469200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/0.log"
Jul 09 04:01:23.469246 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.469238    2577 generic.go:356] "Generic (PLEG): container finished" podID="d7de353e-7173-43ab-879e-97b27640cbc6" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d" exitCode=255
Jul 09 04:01:23.469350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.469324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" event={"ID":"d7de353e-7173-43ab-879e-97b27640cbc6","Type":"ContainerDied","Data":"9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"}
Jul 09 04:01:23.469400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.469375    2577 scope.go:117] "RemoveContainer" containerID="4bae03f81189533bb54b7339acc3fd04355712d23336e23425c4582185eceb2e"
Jul 09 04:01:23.469599 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.469584    2577 scope.go:117] "RemoveContainer" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"
Jul 09 04:01:23.469789 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:23.469773    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:01:23.505923 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.505894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:23.645661 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.645629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:23.645865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:23.645688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:23.645865 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:23.645758    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:23.645865 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:23.645771    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 04:01:23.645865 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:23.645826    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:25.645806099 +0000 UTC m=+84.920184005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:23.645865 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:23.645847    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:25.645837781 +0000 UTC m=+84.920215697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : configmap "kube-rbac-proxy" not found
Jul 09 04:01:24.011576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.011537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:24.090876 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.090848    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xbljd_29bad720-862e-46d4-a5e6-39b7fe607ff6/dns-node-resolver/0.log"
Jul 09 04:01:24.112058 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.112026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:24.475208 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.475179    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/1.log"
Jul 09 04:01:24.475570 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.475485    2577 scope.go:117] "RemoveContainer" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"
Jul 09 04:01:24.475633 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:24.475614    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:01:24.633524 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:24.633492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:25.091790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.091767    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xmpxf_c8d6e6e9-c491-40bf-b405-df699fc273b6/node-ca/0.log"
Jul 09 04:01:25.491598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.491566    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bf7hv_4eff9c62-944c-454a-8dd3-830e3fcc7c23/serve-healthcheck-canary/0.log"
Jul 09 04:01:25.618230 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.618197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:25.660360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.660333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:25.660504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.660381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:25.660504 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:25.660453    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 04:01:25.660504 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:25.660484    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:25.660600 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:25.660521    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:29.660505938 +0000 UTC m=+88.934883824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : configmap "kube-rbac-proxy" not found
Jul 09 04:01:25.660600 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:25.660565    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:29.660558226 +0000 UTC m=+88.934936112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:25.719741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:25.719716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:26.354798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:26.354762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:26.365738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:26.365714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:26.365876 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.365863    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 04:01:26.365930 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.365920    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls podName:9c041576-2b96-42f5-a4eb-2744f06d267f nodeName:}" failed. No retries permitted until 2026-07-09 04:01:34.365904421 +0000 UTC m=+93.640282313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-scnwx" (UID: "9c041576-2b96-42f5-a4eb-2744f06d267f") : secret "samples-operator-tls" not found
Jul 09 04:01:26.419082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:26.419055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:26.466797 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:26.466765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:26.466907 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.466886    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:26.466958 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.466947    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:34.466930369 +0000 UTC m=+93.741308288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:26.769452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:26.769369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:26.769834 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.769514    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 04:01:26.769834 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.769527    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6877dd5876-glhmh: secret "image-registry-tls" not found
Jul 09 04:01:26.769834 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:26.769575    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls podName:66a3b7c6-3e76-4ba0-9998-0c765ed326fc nodeName:}" failed. No retries permitted until 2026-07-09 04:01:34.769562059 +0000 UTC m=+94.043939945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls") pod "image-registry-6877dd5876-glhmh" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc") : secret "image-registry-tls" not found
Jul 09 04:01:27.234568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:27.234532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:29.035178 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.035148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:29.035594 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.035196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:29.035594 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.035513    2577 scope.go:117] "RemoveContainer" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"
Jul 09 04:01:29.035685 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:29.035645    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:01:29.691980 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.691947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:29.692117 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.692054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:29.692117 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:29.692071    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 04:01:29.692202 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:29.692150    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:37.692129844 +0000 UTC m=+96.966507749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : configmap "kube-rbac-proxy" not found
Jul 09 04:01:29.692202 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:29.692152    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:29.692202 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:29.692188    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:37.692178423 +0000 UTC m=+96.966556309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:29.891231 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.891194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:29.989898 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:29.989819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:30.090988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:30.090949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:30.194737 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:30.194709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:30.291572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:30.291489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:30.722518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:30.722486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:30.786485 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:30.786448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:31.559943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:31.559916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:31.950148 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:31.950116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:32.159504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:32.159454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:32.350139 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:32.350099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:32.755205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:32.755101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:33.157973 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:33.157918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:33.548807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:33.548712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:33.605783 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:33.605748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.152760 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.152729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.253867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.253831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.431591 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.431505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:34.433867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.433839    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c041576-2b96-42f5-a4eb-2744f06d267f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-scnwx\" (UID: \"9c041576-2b96-42f5-a4eb-2744f06d267f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:34.526028 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.526002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"
Jul 09 04:01:34.532138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.532117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:34.532237 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:34.532225    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 04:01:34.532292 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:34.532276    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert podName:04793198-4033-4011-bebf-d70b4b2b8185 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:50.532258603 +0000 UTC m=+109.806636489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z2jn5" (UID: "04793198-4033-4011-bebf-d70b4b2b8185") : secret "networking-console-plugin-cert" not found
Jul 09 04:01:34.552741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.552716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.642396 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.642365    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx"]
Jul 09 04:01:34.763945 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.763918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.833368 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.833342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:34.835376 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.835359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:34.837541 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.837526    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"image-registry-6877dd5876-glhmh\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") " pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:34.905669 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:34.905641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:35.025695 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.025667    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:01:35.028408 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:35.028372    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a3b7c6_3e76_4ba0_9998_0c765ed326fc.slice/crio-f7bf32b7f6d8df62161454e8c06998127cff23f3236d10a09907df53ec6b026a WatchSource:0}: Error finding container f7bf32b7f6d8df62161454e8c06998127cff23f3236d10a09907df53ec6b026a: Status 404 returned error can't find the container with id f7bf32b7f6d8df62161454e8c06998127cff23f3236d10a09907df53ec6b026a
Jul 09 04:01:35.123329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.123297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:35.364595 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.364547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:35.502626 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.502585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx" event={"ID":"9c041576-2b96-42f5-a4eb-2744f06d267f","Type":"ContainerStarted","Data":"3728081ac92808bca7b0f33eb40ba2b9e36634f1b3132afd087fc1e1a2d33066"}
Jul 09 04:01:35.503976 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.503954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" event={"ID":"66a3b7c6-3e76-4ba0-9998-0c765ed326fc","Type":"ContainerStarted","Data":"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"}
Jul 09 04:01:35.504044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.503993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" event={"ID":"66a3b7c6-3e76-4ba0-9998-0c765ed326fc","Type":"ContainerStarted","Data":"f7bf32b7f6d8df62161454e8c06998127cff23f3236d10a09907df53ec6b026a"}
Jul 09 04:01:35.504100 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.504087    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:35.521971 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.521927    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" podStartSLOduration=17.521913375 podStartE2EDuration="17.521913375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:35.520818942 +0000 UTC m=+94.795196862" watchObservedRunningTime="2026-07-09 04:01:35.521913375 +0000 UTC m=+94.796291283"
Jul 09 04:01:35.949309 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:35.949269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:36.150001 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:36.149962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:36.359660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:36.359626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:36.422422 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:36.422386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:36.557713 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:36.557680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:37.512891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.512857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx" event={"ID":"9c041576-2b96-42f5-a4eb-2744f06d267f","Type":"ContainerStarted","Data":"a2d637529793943fefebfac85e5f2ab45d5e054f8820b149762c330d078660a5"}
Jul 09 04:01:37.513240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.512905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx" event={"ID":"9c041576-2b96-42f5-a4eb-2744f06d267f","Type":"ContainerStarted","Data":"6bdbf266fcea868d924bbe8f6cdb71d30cac03ee3a3ecb8dbabca72488672162"}
Jul 09 04:01:37.530609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.530526    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-scnwx" podStartSLOduration=16.951141143 podStartE2EDuration="19.530511325s" totalImagesPullingTime="2.579370182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:37.529268344 +0000 UTC m=+96.803646272" watchObservedRunningTime="2026-07-09 04:01:37.530511325 +0000 UTC m=+96.804889232"
Jul 09 04:01:37.552362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.552333    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:37.752800 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.752769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:37.756784 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.756764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:37.756884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.756812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:37.756884 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:37.756864    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:37.756951 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:37.756917    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls podName:23a60a31-a3f7-44ae-b5bc-49d8bc0f3516 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:53.756903823 +0000 UTC m=+113.031281710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlt7p" (UID: "23a60a31-a3f7-44ae-b5bc-49d8bc0f3516") : secret "insights-runtime-extractor-tls" not found
Jul 09 04:01:37.757273 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.757258    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:37.948851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:37.948812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:38.350417 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:38.350389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:38.965483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:38.965442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:38.967578 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:38.967556    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4538ad5-df7b-497a-b315-8b49ced9d705-metrics-tls\") pod \"dns-default-wwrtx\" (UID: \"c4538ad5-df7b-497a-b315-8b49ced9d705\") " pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:39.030390 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.030366    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wgmf2\""
Jul 09 04:01:39.039049 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.039027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:39.077002 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.076975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:39.159994 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.159966    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wwrtx"]
Jul 09 04:01:39.162816 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:39.162792    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4538ad5_df7b_497a_b315_8b49ced9d705.slice/crio-db189b772888c0afd23bbdabbeef5659c6e545d965e25df5c7152206474b7ea6 WatchSource:0}: Error finding container db189b772888c0afd23bbdabbeef5659c6e545d965e25df5c7152206474b7ea6: Status 404 returned error can't find the container with id db189b772888c0afd23bbdabbeef5659c6e545d965e25df5c7152206474b7ea6
Jul 09 04:01:39.173288 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.173260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:39.519383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.519349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wwrtx" event={"ID":"c4538ad5-df7b-497a-b315-8b49ced9d705","Type":"ContainerStarted","Data":"db189b772888c0afd23bbdabbeef5659c6e545d965e25df5c7152206474b7ea6"}
Jul 09 04:01:39.556650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:39.556622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:40.174444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.174417    2577 scope.go:117] "RemoveContainer" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"
Jul 09 04:01:40.471411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.471321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:40.524339 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.524309    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:01:40.524812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.524795    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/1.log"
Jul 09 04:01:40.524925 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.524835    2577 generic.go:356] "Generic (PLEG): container finished" podID="d7de353e-7173-43ab-879e-97b27640cbc6" containerID="2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85" exitCode=255
Jul 09 04:01:40.524925 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.524877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" event={"ID":"d7de353e-7173-43ab-879e-97b27640cbc6","Type":"ContainerDied","Data":"2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85"}
Jul 09 04:01:40.525024 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.524949    2577 scope.go:117] "RemoveContainer" containerID="9af18ad11ea8873ffb1132ed4878e68216777205c3f008e612f64b87eecec95d"
Jul 09 04:01:40.525268 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.525249    2577 scope.go:117] "RemoveContainer" containerID="2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85"
Jul 09 04:01:40.525498 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:40.525455    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:01:40.544322 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.544292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:40.601646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:40.601612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:41.434815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.434778    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-986sz"
Jul 09 04:01:41.529207 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.529180    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:01:41.530758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.530729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wwrtx" event={"ID":"c4538ad5-df7b-497a-b315-8b49ced9d705","Type":"ContainerStarted","Data":"4c89de665ded6fa36d1ec93774db523c87d6094feebf6b4ae1f335a94e8d9839"}
Jul 09 04:01:41.530870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.530783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wwrtx" event={"ID":"c4538ad5-df7b-497a-b315-8b49ced9d705","Type":"ContainerStarted","Data":"cc25ec5d7bd2698ce9871e182cb259d09cf3effe47a1413c9028ff44b8da4079"}
Jul 09 04:01:41.530870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.530833    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:41.561075 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:41.561026    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wwrtx" podStartSLOduration=64.924822722 podStartE2EDuration="1m6.561008222s" totalImagesPullingTime="1.6361855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:41.560572019 +0000 UTC m=+100.834949948" watchObservedRunningTime="2026-07-09 04:01:41.561008222 +0000 UTC m=+100.835386130"
Jul 09 04:01:42.133274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:42.133242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:42.229039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:42.229005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:49.035225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.035174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:49.035659 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.035247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:01:49.035659 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.035573    2577 scope.go:117] "RemoveContainer" containerID="2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85"
Jul 09 04:01:49.035733 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:01:49.035715    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:01:49.388665 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.388634    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"]
Jul 09 04:01:49.391522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.391500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.393854 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.393829    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 04:01:49.394241 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.394215    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 04:01:49.394355 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.394326    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 09 04:01:49.394480 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.394326    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 04:01:49.394480 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.394376    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4pf5m\""
Jul 09 04:01:49.402402 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.402380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"]
Jul 09 04:01:49.412095 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.412074    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:01:49.483610 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.483578    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"]
Jul 09 04:01:49.486064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.486046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.503036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.503015    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"]
Jul 09 04:01:49.546214 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.546181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b6a19b6-5e27-48af-9559-bd151668eea2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.546375 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.546241    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qkw7\" (UniqueName: \"kubernetes.io/projected/7b6a19b6-5e27-48af-9559-bd151668eea2-kube-api-access-9qkw7\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.557100 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.557073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:49.646690 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.646596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b6a19b6-5e27-48af-9559-bd151668eea2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.646825 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.646783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qkw7\" (UniqueName: \"kubernetes.io/projected/7b6a19b6-5e27-48af-9559-bd151668eea2-kube-api-access-9qkw7\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.646825 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.646809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzppt\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-kube-api-access-tzppt\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.647240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.646843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-trusted-ca\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.647441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-installation-pull-secrets\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.647656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-bound-sa-token\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.647836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:49.647836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-image-registry-private-configuration\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.648008 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-tls\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.648008 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86fc61f0-2850-482e-8cf4-64a3fbc63529-ca-trust-extracted\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.648008 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.647953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-certificates\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.651237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.650706    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b6a19b6-5e27-48af-9559-bd151668eea2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.657271 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.657249    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qkw7\" (UniqueName: \"kubernetes.io/projected/7b6a19b6-5e27-48af-9559-bd151668eea2-kube-api-access-9qkw7\") pod \"managed-serviceaccount-addon-agent-59957777b-4z8r2\" (UID: \"7b6a19b6-5e27-48af-9559-bd151668eea2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.708150 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.708122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"
Jul 09 04:01:49.748416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-installation-pull-secrets\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-bound-sa-token\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748668 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-image-registry-private-configuration\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748668 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-tls\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748668 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86fc61f0-2850-482e-8cf4-64a3fbc63529-ca-trust-extracted\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748668 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-certificates\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzppt\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-kube-api-access-tzppt\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.748877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.748720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-trusted-ca\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.749677 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.749625    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86fc61f0-2850-482e-8cf4-64a3fbc63529-ca-trust-extracted\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.749800 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.749702    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-trusted-ca\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.749800 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.749724    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-certificates\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.750997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.750969    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-installation-pull-secrets\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.751189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.751172    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-registry-tls\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.751951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.751915    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86fc61f0-2850-482e-8cf4-64a3fbc63529-image-registry-private-configuration\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.757117 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.757073    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-bound-sa-token\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.757485 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.757451    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzppt\" (UniqueName: \"kubernetes.io/projected/86fc61f0-2850-482e-8cf4-64a3fbc63529-kube-api-access-tzppt\") pod \"image-registry-6d5cf85f5f-4bkx5\" (UID: \"86fc61f0-2850-482e-8cf4-64a3fbc63529\") " pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.794838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.794812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:49.829406 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.829368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2"]
Jul 09 04:01:49.833178 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:49.833146    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b6a19b6_5e27_48af_9559_bd151668eea2.slice/crio-548460bd6f26116a66eb12a00419747052b3c30ce4e31a091a2723ab62780914 WatchSource:0}: Error finding container 548460bd6f26116a66eb12a00419747052b3c30ce4e31a091a2723ab62780914: Status 404 returned error can't find the container with id 548460bd6f26116a66eb12a00419747052b3c30ce4e31a091a2723ab62780914
Jul 09 04:01:49.945611 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:49.945581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"]
Jul 09 04:01:49.948732 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:49.948708    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fc61f0_2850_482e_8cf4_64a3fbc63529.slice/crio-d0aa1d12f743515017cfd00e1db7c4d32db5340aba80b69c908f1c479d0864cd WatchSource:0}: Error finding container d0aa1d12f743515017cfd00e1db7c4d32db5340aba80b69c908f1c479d0864cd: Status 404 returned error can't find the container with id d0aa1d12f743515017cfd00e1db7c4d32db5340aba80b69c908f1c479d0864cd
Jul 09 04:01:50.553001 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.552967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:50.555674 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.555648    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04793198-4033-4011-bebf-d70b4b2b8185-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2jn5\" (UID: \"04793198-4033-4011-bebf-d70b4b2b8185\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:50.559992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.559964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2" event={"ID":"7b6a19b6-5e27-48af-9559-bd151668eea2","Type":"ContainerStarted","Data":"548460bd6f26116a66eb12a00419747052b3c30ce4e31a091a2723ab62780914"}
Jul 09 04:01:50.561601 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.561490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5" event={"ID":"86fc61f0-2850-482e-8cf4-64a3fbc63529","Type":"ContainerStarted","Data":"95aebfcf582ddd85d9445b6b24111bd988456b55b178a8c366a1b995931294fc"}
Jul 09 04:01:50.561601 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.561540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5" event={"ID":"86fc61f0-2850-482e-8cf4-64a3fbc63529","Type":"ContainerStarted","Data":"d0aa1d12f743515017cfd00e1db7c4d32db5340aba80b69c908f1c479d0864cd"}
Jul 09 04:01:50.561865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.561825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:01:50.581552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.581412    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5" podStartSLOduration=1.581393185 podStartE2EDuration="1.581393185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:50.580830271 +0000 UTC m=+109.855208180" watchObservedRunningTime="2026-07-09 04:01:50.581393185 +0000 UTC m=+109.855771093"
Jul 09 04:01:50.829804 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.829716    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"
Jul 09 04:01:50.978814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:50.978777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5"]
Jul 09 04:01:50.982329 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:50.982300    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04793198_4033_4011_bebf_d70b4b2b8185.slice/crio-4f8bfd20b5e437585c7d8f5d37e886d1a03c352691b6ba3934d94c4628bfa6b4 WatchSource:0}: Error finding container 4f8bfd20b5e437585c7d8f5d37e886d1a03c352691b6ba3934d94c4628bfa6b4: Status 404 returned error can't find the container with id 4f8bfd20b5e437585c7d8f5d37e886d1a03c352691b6ba3934d94c4628bfa6b4
Jul 09 04:01:51.535893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:51.535864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wwrtx"
Jul 09 04:01:51.568027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:51.567945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5" event={"ID":"04793198-4033-4011-bebf-d70b4b2b8185","Type":"ContainerStarted","Data":"4f8bfd20b5e437585c7d8f5d37e886d1a03c352691b6ba3934d94c4628bfa6b4"}
Jul 09 04:01:52.573057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:52.572965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2" event={"ID":"7b6a19b6-5e27-48af-9559-bd151668eea2","Type":"ContainerStarted","Data":"cf9ade97f06e6169395fa0ef1eaa76794a8a3abe75ed71bcae0f0b485e0d6e89"}
Jul 09 04:01:52.586041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:52.585992    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59957777b-4z8r2" podStartSLOduration=1.182100805 podStartE2EDuration="3.585976404s" totalImagesPullingTime="2.403875599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:52.584972163 +0000 UTC m=+111.859350095" watchObservedRunningTime="2026-07-09 04:01:52.585976404 +0000 UTC m=+111.860354656"
Jul 09 04:01:53.577201 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.577161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5" event={"ID":"04793198-4033-4011-bebf-d70b4b2b8185","Type":"ContainerStarted","Data":"fa0965b0825cd758acdfb46a769388f2132516b541d3b3d55149d7ba15d6de54"}
Jul 09 04:01:53.592955 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.592909    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2jn5" podStartSLOduration=33.612396565 podStartE2EDuration="35.592892436s" totalImagesPullingTime="1.980495871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:53.592386211 +0000 UTC m=+112.866764149" watchObservedRunningTime="2026-07-09 04:01:53.592892436 +0000 UTC m=+112.867270344"
Jul 09 04:01:53.710677 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.710634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:53.780763 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.780725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:53.783066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.783044    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/23a60a31-a3f7-44ae-b5bc-49d8bc0f3516-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlt7p\" (UID: \"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516\") " pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:53.811928 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.811889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:53.922128 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.922088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:53.988705 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:53.988672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:54.063640 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:54.063608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlt7p"
Jul 09 04:01:54.190313 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:54.190235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlt7p"]
Jul 09 04:01:54.193306 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:01:54.193281    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a60a31_a3f7_44ae_b5bc_49d8bc0f3516.slice/crio-36ce97fc1eeb9623ff61ef211e0f462085cbf41c315e1283790cf62fa05c7544 WatchSource:0}: Error finding container 36ce97fc1eeb9623ff61ef211e0f462085cbf41c315e1283790cf62fa05c7544: Status 404 returned error can't find the container with id 36ce97fc1eeb9623ff61ef211e0f462085cbf41c315e1283790cf62fa05c7544
Jul 09 04:01:54.581974 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:54.581893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlt7p" event={"ID":"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516","Type":"ContainerStarted","Data":"36ce97fc1eeb9623ff61ef211e0f462085cbf41c315e1283790cf62fa05c7544"}
Jul 09 04:01:54.650483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:54.650449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:58.534775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:58.534743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:58.602332 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:58.602294    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlt7p" event={"ID":"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516","Type":"ContainerStarted","Data":"766cc975576d83660b5622f5d966018f7a2e061076fdae820ca1259edd86fa31"}
Jul 09 04:01:58.636851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:58.636755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:59.418273 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:59.418204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:01:59.606764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:59.606728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlt7p" event={"ID":"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516","Type":"ContainerStarted","Data":"450bf30b3b5e88f76c60810e0462f76640c2a93a9a9555e3561a24468a29061e"}
Jul 09 04:01:59.917053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:59.917017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:59.976122 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:01:59.976087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:00.174423 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:00.174350    2577 scope.go:117] "RemoveContainer" containerID="2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85"
Jul 09 04:02:00.174605 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:00.174579    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2vxxm_openshift-console-operator(d7de353e-7173-43ab-879e-97b27640cbc6)\"" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podUID="d7de353e-7173-43ab-879e-97b27640cbc6"
Jul 09 04:02:00.495888 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:00.495799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:00.626880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:00.626817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:00.683905 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:00.683868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:02.619361 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:02.619326    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlt7p" event={"ID":"23a60a31-a3f7-44ae-b5bc-49d8bc0f3516","Type":"ContainerStarted","Data":"efc576534e697d9819e58fbd320721d1ae348fa98ddaad39ce2dbe78a08853d7"}
Jul 09 04:02:02.647797 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:02.647733    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vlt7p" podStartSLOduration=34.260577703 podStartE2EDuration="41.647719087s" totalImagesPullingTime="7.387141384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:01:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:02.646714453 +0000 UTC m=+121.921092363" watchObservedRunningTime="2026-07-09 04:02:02.647719087 +0000 UTC m=+121.922096994"
Jul 09 04:02:02.921114 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:02.921025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:03.258945 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:03.258857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:03.312223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:03.312186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:05.054027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:05.053993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:05.264495 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:05.264442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:05.824638 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:05.824604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:05.925143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:05.925113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:06.738323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:06.738287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:06.839483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:06.839430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:06.940414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:06.940384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:07.544996 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:07.544958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:07.645968 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:07.645937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:08.735977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.735941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:08.761299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.761269    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"]
Jul 09 04:02:08.764688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.764673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.766627 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.766597    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 04:02:08.766749 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.766713    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 04:02:08.766870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.766857    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 04:02:08.767325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.767306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 04:02:08.767400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.767334    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-btdx5\""
Jul 09 04:02:08.767400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.767309    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 04:02:08.767400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.767380    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 04:02:08.789067 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.789049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"]
Jul 09 04:02:08.861783 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.861746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:08.901782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.901895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07e5f30a-e6c4-485d-85a5-d15885e27d88-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.901895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.901895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.902014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.902014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.901949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vccq\" (UniqueName: \"kubernetes.io/projected/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-api-access-4vccq\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:08.917943 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.917916    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"]
Jul 09 04:02:08.921029 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.921013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:08.922918 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.922902    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 04:02:08.923095 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.923077    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 04:02:08.923251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.923235    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pg2vn\""
Jul 09 04:02:08.932444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.932425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"]
Jul 09 04:02:08.943885 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.943868    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p4gpk"]
Jul 09 04:02:08.946934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.946916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:08.949248 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.949229    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 04:02:08.949342 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.949246    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 04:02:08.949342 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.949287    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2w2m\""
Jul 09 04:02:08.949342 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.949306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 04:02:08.951792 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:08.951768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.002812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07e5f30a-e6c4-485d-85a5-d15885e27d88-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.002812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.002967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.002967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.002967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vccq\" (UniqueName: \"kubernetes.io/projected/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-api-access-4vccq\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.002967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.002957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.003154 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.003139    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07e5f30a-e6c4-485d-85a5-d15885e27d88-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.003512 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.003488    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.003612 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.003542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07e5f30a-e6c4-485d-85a5-d15885e27d88-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.005330 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.005309    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.005416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.005399    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.016046 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.016028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vccq\" (UniqueName: \"kubernetes.io/projected/07e5f30a-e6c4-485d-85a5-d15885e27d88-kube-api-access-4vccq\") pod \"kube-state-metrics-7c9dbb8b7d-6fvb4\" (UID: \"07e5f30a-e6c4-485d-85a5-d15885e27d88\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.074171 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.074152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"
Jul 09 04:02:09.104171 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b209ea7b-6073-42b4-aaeb-679f5df5c42c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.104315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-textfile\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.104315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ttpq\" (UniqueName: \"kubernetes.io/projected/b209ea7b-6073-42b4-aaeb-679f5df5c42c-kube-api-access-2ttpq\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.104315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-root\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-metrics-client-ca\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-sys\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-wtmp\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-accelerators-collector-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgzl\" (UniqueName: \"kubernetes.io/projected/ff71642a-d812-44ad-b774-38d253a9c03e-kube-api-access-6lgzl\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.104774 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.104578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.121414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.121384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.201769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.201741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4"]
Jul 09 04:02:09.204506 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:09.204461    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e5f30a_e6c4_485d_85a5_d15885e27d88.slice/crio-11db632b0993a231a42ccb6b171932dcc90807072c7465335979ce256091c0fe WatchSource:0}: Error finding container 11db632b0993a231a42ccb6b171932dcc90807072c7465335979ce256091c0fe: Status 404 returned error can't find the container with id 11db632b0993a231a42ccb6b171932dcc90807072c7465335979ce256091c0fe
Jul 09 04:02:09.205454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.205576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-sys\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-wtmp\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205719 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-accelerators-collector-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205719 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205623    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-sys\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205739    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-wtmp\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgzl\" (UniqueName: \"kubernetes.io/projected/ff71642a-d812-44ad-b774-38d253a9c03e-kube-api-access-6lgzl\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b209ea7b-6073-42b4-aaeb-679f5df5c42c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.205966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-textfile\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.205966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.205966 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:09.205942    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 04:02:09.206157 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:09.205994    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls podName:ff71642a-d812-44ad-b774-38d253a9c03e nodeName:}" failed. No retries permitted until 2026-07-09 04:02:09.705977132 +0000 UTC m=+128.980355021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls") pod "node-exporter-p4gpk" (UID: "ff71642a-d812-44ad-b774-38d253a9c03e") : secret "node-exporter-tls" not found
Jul 09 04:02:09.206157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.205967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ttpq\" (UniqueName: \"kubernetes.io/projected/b209ea7b-6073-42b4-aaeb-679f5df5c42c-kube-api-access-2ttpq\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.206157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-root\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.206483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-metrics-client-ca\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.206483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206266    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-accelerators-collector-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.206483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206287    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-textfile\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.206773 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206754    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff71642a-d812-44ad-b774-38d253a9c03e-metrics-client-ca\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.206860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.206819    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff71642a-d812-44ad-b774-38d253a9c03e-root\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.207040 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.207018    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b209ea7b-6073-42b4-aaeb-679f5df5c42c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.207639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.207611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.208150 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.208127    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.208262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.208235    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.208965 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.208946    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b209ea7b-6073-42b4-aaeb-679f5df5c42c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.212721 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.212703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgzl\" (UniqueName: \"kubernetes.io/projected/ff71642a-d812-44ad-b774-38d253a9c03e-kube-api-access-6lgzl\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.213345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.213326    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ttpq\" (UniqueName: \"kubernetes.io/projected/b209ea7b-6073-42b4-aaeb-679f5df5c42c-kube-api-access-2ttpq\") pod \"openshift-state-metrics-54cf96dfcf-mnzc2\" (UID: \"b209ea7b-6073-42b4-aaeb-679f5df5c42c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.230440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.230425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"
Jul 09 04:02:09.349829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.349800    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2"]
Jul 09 04:02:09.352971 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:09.352946    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb209ea7b_6073_42b4_aaeb_679f5df5c42c.slice/crio-a93ead52bfba5d965b8628eed4bec35bf74a82a507f443c4af26a17545a4d53a WatchSource:0}: Error finding container a93ead52bfba5d965b8628eed4bec35bf74a82a507f443c4af26a17545a4d53a: Status 404 returned error can't find the container with id a93ead52bfba5d965b8628eed4bec35bf74a82a507f443c4af26a17545a4d53a
Jul 09 04:02:09.566780 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.566689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.642700 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.642656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2" event={"ID":"b209ea7b-6073-42b4-aaeb-679f5df5c42c","Type":"ContainerStarted","Data":"ac8f206b5df2ffa3dbf9c54241d007fb395ccb8033649318e23168e11a8133b6"}
Jul 09 04:02:09.642865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.642727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2" event={"ID":"b209ea7b-6073-42b4-aaeb-679f5df5c42c","Type":"ContainerStarted","Data":"78e37f228c271b006cf47d5ebf777566b41b7bef31e2e8693a80cf4b45e25ca2"}
Jul 09 04:02:09.642865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.642752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2" event={"ID":"b209ea7b-6073-42b4-aaeb-679f5df5c42c","Type":"ContainerStarted","Data":"a93ead52bfba5d965b8628eed4bec35bf74a82a507f443c4af26a17545a4d53a"}
Jul 09 04:02:09.643783 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.643760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4" event={"ID":"07e5f30a-e6c4-485d-85a5-d15885e27d88","Type":"ContainerStarted","Data":"11db632b0993a231a42ccb6b171932dcc90807072c7465335979ce256091c0fe"}
Jul 09 04:02:09.703194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.703129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.711133 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.711113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.713728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.713708    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff71642a-d812-44ad-b774-38d253a9c03e-node-exporter-tls\") pod \"node-exporter-p4gpk\" (UID: \"ff71642a-d812-44ad-b774-38d253a9c03e\") " pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.811386 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.811304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.856077 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.856053    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p4gpk"
Jul 09 04:02:09.866737 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:09.866703    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff71642a_d812_44ad_b774_38d253a9c03e.slice/crio-f62f31f451c5abba427af663b8b72105d9836f3a69a011c89a378e24afa131c1 WatchSource:0}: Error finding container f62f31f451c5abba427af663b8b72105d9836f3a69a011c89a378e24afa131c1: Status 404 returned error can't find the container with id f62f31f451c5abba427af663b8b72105d9836f3a69a011c89a378e24afa131c1
Jul 09 04:02:09.904094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.904068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:09.913014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.912993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:02:09.915681 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.915661    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba58ffef-ea23-486c-ad2e-be920dc8173c-metrics-certs\") pod \"network-metrics-daemon-st6ww\" (UID: \"ba58ffef-ea23-486c-ad2e-be920dc8173c\") " pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:02:09.925539 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.925515    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:02:09.930404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.930381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:09.932259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932179    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 04:02:09.932365 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 04:02:09.932444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932429    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 04:02:09.932522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 04:02:09.932522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jjz8f\""
Jul 09 04:02:09.932627 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 04:02:09.932784 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 04:02:09.932838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 04:02:09.932838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932811    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 04:02:09.932946 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.932768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 04:02:09.949012 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.947885    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:02:09.967732 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:09.967703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:10.100666 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.100632    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fn82r\""
Jul 09 04:02:10.108411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.108346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-st6ww"
Jul 09 04:02:10.115448 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djxs8\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.115839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.116128 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.115884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.216912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.216912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.216912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.216912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.216973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djxs8\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.217714 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.217655    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.218446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.218414    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.219354 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.219323    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.219564 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:10.219538    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 09 04:02:10.219682 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:10.219665    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls podName:6bf449ab-0cbe-459c-8cb9-bda5aab7cc45 nodeName:}" failed. No retries permitted until 2026-07-09 04:02:10.719643616 +0000 UTC m=+129.994021517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45") : secret "alertmanager-main-tls" not found
Jul 09 04:02:10.220110 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.220088    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.220776 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.220720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.221251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.221190    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.221682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.221659    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.221770 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.221743    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.222922 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.222898    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.223657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.223421    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.224133 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.224112    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.225876 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.225854    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djxs8\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.527813 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.527721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:10.649224 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.649178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p4gpk" event={"ID":"ff71642a-d812-44ad-b774-38d253a9c03e","Type":"ContainerStarted","Data":"f62f31f451c5abba427af663b8b72105d9836f3a69a011c89a378e24afa131c1"}
Jul 09 04:02:10.721175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.721146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.724530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.723822    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:10.804984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.804908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-st6ww"]
Jul 09 04:02:10.809800 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:10.809498    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba58ffef_ea23_486c_ad2e_be920dc8173c.slice/crio-8f6b614ca0af4e211e719abe9d433222640810155b0cc31536904d28510f2a28 WatchSource:0}: Error finding container 8f6b614ca0af4e211e719abe9d433222640810155b0cc31536904d28510f2a28: Status 404 returned error can't find the container with id 8f6b614ca0af4e211e719abe9d433222640810155b0cc31536904d28510f2a28
Jul 09 04:02:10.841119 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:10.841026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:02:11.037105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.036968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:02:11.213318 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:11.213277    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf449ab_0cbe_459c_8cb9_bda5aab7cc45.slice/crio-1710a10bec6e0c6f951a7bc3c3773d8ef9eca4c7fb7045f8f66c68f9984fb281 WatchSource:0}: Error finding container 1710a10bec6e0c6f951a7bc3c3773d8ef9eca4c7fb7045f8f66c68f9984fb281: Status 404 returned error can't find the container with id 1710a10bec6e0c6f951a7bc3c3773d8ef9eca4c7fb7045f8f66c68f9984fb281
Jul 09 04:02:11.574091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.574020    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d5cf85f5f-4bkx5"
Jul 09 04:02:11.658709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.658654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-st6ww" event={"ID":"ba58ffef-ea23-486c-ad2e-be920dc8173c","Type":"ContainerStarted","Data":"8f6b614ca0af4e211e719abe9d433222640810155b0cc31536904d28510f2a28"}
Jul 09 04:02:11.661557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.661507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2" event={"ID":"b209ea7b-6073-42b4-aaeb-679f5df5c42c","Type":"ContainerStarted","Data":"c9a5979daad0c80dfe93c8aad4e0e87de163fdaa8b163a7fcf989f5677fa9b27"}
Jul 09 04:02:11.663609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.663584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4" event={"ID":"07e5f30a-e6c4-485d-85a5-d15885e27d88","Type":"ContainerStarted","Data":"2a616e78ae885f5dee737380137761098867e2b548a483e01e2c73355c87df2e"}
Jul 09 04:02:11.663715 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.663624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4" event={"ID":"07e5f30a-e6c4-485d-85a5-d15885e27d88","Type":"ContainerStarted","Data":"62fad47be46819e74d4e7c7a068b1115db73872b81566984d325a40bf5bca74f"}
Jul 09 04:02:11.663715 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.663644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4" event={"ID":"07e5f30a-e6c4-485d-85a5-d15885e27d88","Type":"ContainerStarted","Data":"4c35222e8f3c021dda6efe0e54ef58ac18b9ad8061d7cacaa8c5c6f50f4e2039"}
Jul 09 04:02:11.665146 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.665121    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff71642a-d812-44ad-b774-38d253a9c03e" containerID="bb5911a56250709b5916aab6dd55e718fdb5f4a5c03f010899be43f5e8631817" exitCode=0
Jul 09 04:02:11.665267 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.665193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p4gpk" event={"ID":"ff71642a-d812-44ad-b774-38d253a9c03e","Type":"ContainerDied","Data":"bb5911a56250709b5916aab6dd55e718fdb5f4a5c03f010899be43f5e8631817"}
Jul 09 04:02:11.666693 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.666663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"1710a10bec6e0c6f951a7bc3c3773d8ef9eca4c7fb7045f8f66c68f9984fb281"}
Jul 09 04:02:11.680025 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.679975    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mnzc2" podStartSLOduration=2.433611763 podStartE2EDuration="3.679958903s" totalImagesPullingTime="1.24634714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:11.678931589 +0000 UTC m=+130.953309517" watchObservedRunningTime="2026-07-09 04:02:11.679958903 +0000 UTC m=+130.954336811"
Jul 09 04:02:11.696848 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.696801    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-6fvb4" podStartSLOduration=2.187205435 podStartE2EDuration="3.696783423s" totalImagesPullingTime="1.509577988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:11.696162794 +0000 UTC m=+130.970540705" watchObservedRunningTime="2026-07-09 04:02:11.696783423 +0000 UTC m=+130.971161333"
Jul 09 04:02:11.726190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.726145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:11.790871 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.790834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:11.890869 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:11.890831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:12.023016 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.022995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:12.130854 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.130825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:12.211597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.211560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:12.673109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.673069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-st6ww" event={"ID":"ba58ffef-ea23-486c-ad2e-be920dc8173c","Type":"ContainerStarted","Data":"c1f2bc7ad584fe9d8cee65ee412175fbcd4a2794e146bfab314e6eed8be84143"}
Jul 09 04:02:12.673297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.673138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-st6ww" event={"ID":"ba58ffef-ea23-486c-ad2e-be920dc8173c","Type":"ContainerStarted","Data":"fc36212ae2b48727ff9336cc5873334bde09c1f8dbebdcacfacbcb8856cac856"}
Jul 09 04:02:12.675443 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.675411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p4gpk" event={"ID":"ff71642a-d812-44ad-b774-38d253a9c03e","Type":"ContainerStarted","Data":"5e08dc8f9112777b9d393e9dff32401083e7718271110ae32972ba73248aae31"}
Jul 09 04:02:12.675597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.675462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p4gpk" event={"ID":"ff71642a-d812-44ad-b774-38d253a9c03e","Type":"ContainerStarted","Data":"ae6741c98cfc445670c5515e06cf98dc7743c3df3346167368694f2e3b9190f4"}
Jul 09 04:02:12.692804 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.692754    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-st6ww" podStartSLOduration=130.470860286 podStartE2EDuration="2m11.692735839s" totalImagesPullingTime="1.221875553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:12.691623441 +0000 UTC m=+131.966001378" watchObservedRunningTime="2026-07-09 04:02:12.692735839 +0000 UTC m=+131.967113750"
Jul 09 04:02:12.717949 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.717887    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p4gpk" podStartSLOduration=3.261450619 podStartE2EDuration="4.7178691s" totalImagesPullingTime="1.390465573s" totalInitContainerRuntime="65.952908ms" isStatefulPod=true podCreationTimestamp="2026-07-09 04:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:12.716485904 +0000 UTC m=+131.990863832" watchObservedRunningTime="2026-07-09 04:02:12.7178691 +0000 UTC m=+131.992247009"
Jul 09 04:02:12.799451 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:12.799418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:13.362977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:13.362911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:13.638059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:13.638029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:13.680895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:13.680864    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4" exitCode=0
Jul 09 04:02:13.681048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:13.680956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4"}
Jul 09 04:02:13.866724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:13.866656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:14.431855 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.431817    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" podUID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" containerName="registry" containerID="cri-o://7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb" gracePeriod=30
Jul 09 04:02:14.674809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.674784    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:02:14.689901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.689836    2577 generic.go:356] "Generic (PLEG): container finished" podID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" containerID="7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb" exitCode=0
Jul 09 04:02:14.689901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.689886    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6877dd5876-glhmh"
Jul 09 04:02:14.689901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.689896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" event={"ID":"66a3b7c6-3e76-4ba0-9998-0c765ed326fc","Type":"ContainerDied","Data":"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"}
Jul 09 04:02:14.690064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.689960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6877dd5876-glhmh" event={"ID":"66a3b7c6-3e76-4ba0-9998-0c765ed326fc","Type":"ContainerDied","Data":"f7bf32b7f6d8df62161454e8c06998127cff23f3236d10a09907df53ec6b026a"}
Jul 09 04:02:14.690064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.689983    2577 scope.go:117] "RemoveContainer" containerID="7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"
Jul 09 04:02:14.701635 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.701616    2577 scope.go:117] "RemoveContainer" containerID="7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"
Jul 09 04:02:14.701920 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:02:14.701900    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb\": container with ID starting with 7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb not found: ID does not exist" containerID="7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"
Jul 09 04:02:14.702010 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.701926    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb"} err="failed to get container status \"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb\": rpc error: code = NotFound desc = could not find container \"7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb\": container with ID starting with 7f47f5e33474aa6766edca15518c2891b6a26a643196425c9237e8f1ccc5f7cb not found: ID does not exist"
Jul 09 04:02:14.750769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.750748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.750889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.750797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.750889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.750829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.750889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.750866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.751055 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751020    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.751109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.751161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf5k7\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.751161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") pod \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\" (UID: \"66a3b7c6-3e76-4ba0-9998-0c765ed326fc\") "
Jul 09 04:02:14.751253 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751205    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:02:14.751307 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751263    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:02:14.751415 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751395    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-trusted-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.751552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.751422    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-certificates\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.753654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.753625    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:02:14.753654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.753638    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:02:14.753767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.753658    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:02:14.753767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.753723    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:02:14.753767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.753751    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7" (OuterVolumeSpecName: "kube-api-access-hf5k7") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "kube-api-access-hf5k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:02:14.759798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.759778    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "66a3b7c6-3e76-4ba0-9998-0c765ed326fc" (UID: "66a3b7c6-3e76-4ba0-9998-0c765ed326fc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:02:14.851735 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851710    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-ca-trust-extracted\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.851735 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851732    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-image-registry-private-configuration\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.851735 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851743    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-installation-pull-secrets\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.851978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851752    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf5k7\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-kube-api-access-hf5k7\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.851978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851761    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-registry-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:14.851978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:14.851773    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66a3b7c6-3e76-4ba0-9998-0c765ed326fc-bound-sa-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:02:15.018280 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.018257    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:02:15.022368 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.022344    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6877dd5876-glhmh"]
Jul 09 04:02:15.117127 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.117093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:15.174051 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.174015    2577 scope.go:117] "RemoveContainer" containerID="2aab08739ec6f12c7c9cbf437f930baaa21fc2c446c2d28e676f817a60224e85"
Jul 09 04:02:15.178845 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.178819    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" path="/var/lib/kubelet/pods/66a3b7c6-3e76-4ba0-9998-0c765ed326fc/volumes"
Jul 09 04:02:15.695622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.695597    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:02:15.696061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.695679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" event={"ID":"d7de353e-7173-43ab-879e-97b27640cbc6","Type":"ContainerStarted","Data":"527d64196088ccf2a5a6799b292028de9f6381ab93abc368ed5d354d6b9d2ba3"}
Jul 09 04:02:15.696061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.695917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:02:15.711503 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.711433    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm" podStartSLOduration=55.153026873 podStartE2EDuration="57.711417822s" totalImagesPullingTime="2.558390949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:15.710098327 +0000 UTC m=+134.984476235" watchObservedRunningTime="2026-07-09 04:02:15.711417822 +0000 UTC m=+134.985795731"
Jul 09 04:02:15.839254 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:15.839226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-2vxxm"
Jul 09 04:02:16.010653 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.010560    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-87d9v"]
Jul 09 04:02:16.010965 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.010945    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" containerName="registry"
Jul 09 04:02:16.010965 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.010964    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" containerName="registry"
Jul 09 04:02:16.011122 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.011009    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a3b7c6-3e76-4ba0-9998-0c765ed326fc" containerName="registry"
Jul 09 04:02:16.015033 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.015014    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:16.016743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.016721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 04:02:16.016856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.016824    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xgfb8\""
Jul 09 04:02:16.016946 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.016932    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 04:02:16.024689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.024668    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-87d9v"]
Jul 09 04:02:16.041338 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.041315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.060401 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.060380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95ng\" (UniqueName: \"kubernetes.io/projected/cbcf6b76-9a1f-4961-a062-4f0eb81c3a72-kube-api-access-j95ng\") pod \"downloads-685bfb5f64-87d9v\" (UID: \"cbcf6b76-9a1f-4961-a062-4f0eb81c3a72\") " pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:16.097259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.097231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.160751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.160727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95ng\" (UniqueName: \"kubernetes.io/projected/cbcf6b76-9a1f-4961-a062-4f0eb81c3a72-kube-api-access-j95ng\") pod \"downloads-685bfb5f64-87d9v\" (UID: \"cbcf6b76-9a1f-4961-a062-4f0eb81c3a72\") " pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:16.167711 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.167677    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95ng\" (UniqueName: \"kubernetes.io/projected/cbcf6b76-9a1f-4961-a062-4f0eb81c3a72-kube-api-access-j95ng\") pod \"downloads-685bfb5f64-87d9v\" (UID: \"cbcf6b76-9a1f-4961-a062-4f0eb81c3a72\") " pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:16.197602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.197579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.326846 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.326828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:16.338201 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.338171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.465819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.465794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-87d9v"]
Jul 09 04:02:16.468462 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:02:16.468423    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcf6b76_9a1f_4961_a062_4f0eb81c3a72.slice/crio-24c120e71271c3c5e3215d501b5882fef11bd486dfe17567b674f26ae20c927a WatchSource:0}: Error finding container 24c120e71271c3c5e3215d501b5882fef11bd486dfe17567b674f26ae20c927a: Status 404 returned error can't find the container with id 24c120e71271c3c5e3215d501b5882fef11bd486dfe17567b674f26ae20c927a
Jul 09 04:02:16.493463 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.493426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.549191 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.549126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.695092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.695037    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.704378 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.704348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"}
Jul 09 04:02:16.704755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.704457    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"}
Jul 09 04:02:16.704755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.704525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"}
Jul 09 04:02:16.704755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.704566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"}
Jul 09 04:02:16.704755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.704594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"}
Jul 09 04:02:16.705543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.705523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-87d9v" event={"ID":"cbcf6b76-9a1f-4961-a062-4f0eb81c3a72","Type":"ContainerStarted","Data":"24c120e71271c3c5e3215d501b5882fef11bd486dfe17567b674f26ae20c927a"}
Jul 09 04:02:16.749282 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.749255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:16.919755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:16.919716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:17.020794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.020734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:17.122108 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.122070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:17.707650 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.707616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:17.713722 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.713695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerStarted","Data":"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"}
Jul 09 04:02:17.740643 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.740601    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.535244636 podStartE2EDuration="8.740582591s" totalImagesPullingTime="5.190320965s" totalInitContainerRuntime="15.01699ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:17.738348838 +0000 UTC m=+137.012726745" watchObservedRunningTime="2026-07-09 04:02:17.740582591 +0000 UTC m=+137.014960500"
Jul 09 04:02:17.909059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:17.909025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:18.212933 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:18.212901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:19.129807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:19.129757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:19.230811 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:19.230777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:22.116194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:22.116159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:22.183730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:22.183697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:27.113791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:27.113752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:27.325845 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:27.325791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:30.528162 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:30.528106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:33.526776 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:33.526738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:33.598080 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:33.598045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:36.331851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:36.331816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:37.796125 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:37.796084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-87d9v" event={"ID":"cbcf6b76-9a1f-4961-a062-4f0eb81c3a72","Type":"ContainerStarted","Data":"0ad00e05938249c6dfcc7ed8e83cfe8dda268b939f22ebfeabd618e58c00c378"}
Jul 09 04:02:37.796635 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:37.796274    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:37.808023 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:37.808002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-87d9v"
Jul 09 04:02:37.812567 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:37.812515    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-87d9v" podStartSLOduration=2.355687242 podStartE2EDuration="22.812494452s" totalImagesPullingTime="20.45680721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:37.810846487 +0000 UTC m=+157.085224416" watchObservedRunningTime="2026-07-09 04:02:37.812494452 +0000 UTC m=+157.086872394"
Jul 09 04:02:40.128911 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:40.128871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:40.564539 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:40.564434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:40.614405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:40.614370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:41.028416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:41.028372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:41.084362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:41.084329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:41.751754 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:41.751701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:41.852676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:41.852640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:42.654195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:42.654156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:42.707648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:42.707612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:52.856727 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:52.856651    2577 generic.go:356] "Generic (PLEG): container finished" podID="304065f4-8a85-4032-87cc-6eedf9065811" containerID="ada533f880e1a5fbaf40226fc68eeda639575ec3b4f95505d504782980e04cae" exitCode=0
Jul 09 04:02:52.856727 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:52.856694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp" event={"ID":"304065f4-8a85-4032-87cc-6eedf9065811","Type":"ContainerDied","Data":"ada533f880e1a5fbaf40226fc68eeda639575ec3b4f95505d504782980e04cae"}
Jul 09 04:02:52.857199 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:52.856996    2577 scope.go:117] "RemoveContainer" containerID="ada533f880e1a5fbaf40226fc68eeda639575ec3b4f95505d504782980e04cae"
Jul 09 04:02:53.862175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:53.862141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-txlvp" event={"ID":"304065f4-8a85-4032-87cc-6eedf9065811","Type":"ContainerStarted","Data":"9347a670c8cf81e8fa8ce319dae4b1a3c5d7bec7adfdc6d798ffc71f8c9a5b77"}
Jul 09 04:02:57.419822 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:57.419781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:02:57.471278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:02:57.471235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:14.792761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:14.792718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:14.849041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:14.849005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:28.094908 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:28.094870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:28.316174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:28.316135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:29.149432 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149395    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:29.149863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149698    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="alertmanager" containerID="cri-o://aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d" gracePeriod=120
Jul 09 04:03:29.149863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149764    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-web" containerID="cri-o://ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947" gracePeriod=120
Jul 09 04:03:29.149863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149765    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-metric" containerID="cri-o://be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521" gracePeriod=120
Jul 09 04:03:29.149863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149810    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="prom-label-proxy" containerID="cri-o://b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1" gracePeriod=120
Jul 09 04:03:29.150078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149845    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy" containerID="cri-o://50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c" gracePeriod=120
Jul 09 04:03:29.150078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:29.149915    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="config-reloader" containerID="cri-o://6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081" gracePeriod=120
Jul 09 04:03:30.002648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002618    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1" exitCode=0
Jul 09 04:03:30.002648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002643    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521" exitCode=0
Jul 09 04:03:30.002648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002650    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c" exitCode=0
Jul 09 04:03:30.002648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002655    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081" exitCode=0
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002660    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d" exitCode=0
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"}
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"}
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"}
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"}
Jul 09 04:03:30.002919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.002846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"}
Jul 09 04:03:30.399099 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.399075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:30.491262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491264    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491302    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491327    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491443    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491495    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djxs8\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491567    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.491775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491641    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume\") pod \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\" (UID: \"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45\") "
Jul 09 04:03:30.492006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491792    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:03:30.492006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491922    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-metrics-client-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.492006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491927    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:03:30.492006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.491942    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:03:30.494157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494133    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.494522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494368    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:03:30.494522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494401    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.494759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494732    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.494829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494785    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out" (OuterVolumeSpecName: "config-out") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:03:30.494893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.494835    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume" (OuterVolumeSpecName: "config-volume") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.495093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.495069    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.496196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.496170    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8" (OuterVolumeSpecName: "kube-api-access-djxs8") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "kube-api-access-djxs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:03:30.498189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.498168    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.504858 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.504835    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config" (OuterVolumeSpecName: "web-config") pod "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" (UID: "6bf449ab-0cbe-459c-8cb9-bda5aab7cc45"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592819    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-main-db\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592845    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-cluster-tls-config\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592875    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-web-config\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592884    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592892    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-main-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.592901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592901    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592910    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-tls-assets\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592918    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-config-out\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592927    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592935    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592943    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:30.593183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:30.592952    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djxs8\" (UniqueName: \"kubernetes.io/projected/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45-kube-api-access-djxs8\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:03:31.009880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.009848    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerID="ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947" exitCode=0
Jul 09 04:03:31.010044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.009890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"}
Jul 09 04:03:31.010044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.009921    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.010044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.009971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6bf449ab-0cbe-459c-8cb9-bda5aab7cc45","Type":"ContainerDied","Data":"1710a10bec6e0c6f951a7bc3c3773d8ef9eca4c7fb7045f8f66c68f9984fb281"}
Jul 09 04:03:31.010044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.010004    2577 scope.go:117] "RemoveContainer" containerID="b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"
Jul 09 04:03:31.018244 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.018226    2577 scope.go:117] "RemoveContainer" containerID="be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"
Jul 09 04:03:31.024989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.024971    2577 scope.go:117] "RemoveContainer" containerID="50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"
Jul 09 04:03:31.031542 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.031521    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:31.032121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.032107    2577 scope.go:117] "RemoveContainer" containerID="ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"
Jul 09 04:03:31.035217 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.035197    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:31.038925 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.038900    2577 scope.go:117] "RemoveContainer" containerID="6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"
Jul 09 04:03:31.045456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.045441    2577 scope.go:117] "RemoveContainer" containerID="aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"
Jul 09 04:03:31.052004 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.051989    2577 scope.go:117] "RemoveContainer" containerID="f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4"
Jul 09 04:03:31.058260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.058243    2577 scope.go:117] "RemoveContainer" containerID="b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"
Jul 09 04:03:31.058611 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.058588    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1\": container with ID starting with b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1 not found: ID does not exist" containerID="b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"
Jul 09 04:03:31.058699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.058618    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1"} err="failed to get container status \"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1\": rpc error: code = NotFound desc = could not find container \"b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1\": container with ID starting with b7a05b7ada740001f7bb252179e76401644da882a09332da40c2ffa9b325c8f1 not found: ID does not exist"
Jul 09 04:03:31.058699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.058637    2577 scope.go:117] "RemoveContainer" containerID="be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"
Jul 09 04:03:31.059118 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.059082    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521\": container with ID starting with be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521 not found: ID does not exist" containerID="be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"
Jul 09 04:03:31.059250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059125    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521"} err="failed to get container status \"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521\": rpc error: code = NotFound desc = could not find container \"be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521\": container with ID starting with be28121bdcff3d270be3d7bacf061bbd4c5617f970aad5e6de5002ae6d5ef521 not found: ID does not exist"
Jul 09 04:03:31.059250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059144    2577 scope.go:117] "RemoveContainer" containerID="50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"
Jul 09 04:03:31.059427 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.059406    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c\": container with ID starting with 50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c not found: ID does not exist" containerID="50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"
Jul 09 04:03:31.059497 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059434    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c"} err="failed to get container status \"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c\": rpc error: code = NotFound desc = could not find container \"50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c\": container with ID starting with 50b3bad11f0a2a18206c81f97f7cf6872e4fc60c5ecb07d32c775408e489734c not found: ID does not exist"
Jul 09 04:03:31.059497 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059451    2577 scope.go:117] "RemoveContainer" containerID="ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"
Jul 09 04:03:31.059699 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.059683    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947\": container with ID starting with ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947 not found: ID does not exist" containerID="ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"
Jul 09 04:03:31.059759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059703    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947"} err="failed to get container status \"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947\": rpc error: code = NotFound desc = could not find container \"ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947\": container with ID starting with ea6817d28fc61ff3abb6c9174eb21f230a1a6eff9a0c9f2387417b58f7c8b947 not found: ID does not exist"
Jul 09 04:03:31.059759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059724    2577 scope.go:117] "RemoveContainer" containerID="6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"
Jul 09 04:03:31.059935 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.059917    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081\": container with ID starting with 6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081 not found: ID does not exist" containerID="6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"
Jul 09 04:03:31.059989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059940    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081"} err="failed to get container status \"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081\": rpc error: code = NotFound desc = could not find container \"6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081\": container with ID starting with 6cb11e97726e6b027aff368e76bac8e93c3abf4cccf3322450b85d3d034d0081 not found: ID does not exist"
Jul 09 04:03:31.059989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.059956    2577 scope.go:117] "RemoveContainer" containerID="aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"
Jul 09 04:03:31.060196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060179    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:31.060290 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.060184    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d\": container with ID starting with aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d not found: ID does not exist" containerID="aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"
Jul 09 04:03:31.060344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060298    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d"} err="failed to get container status \"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d\": rpc error: code = NotFound desc = could not find container \"aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d\": container with ID starting with aa026f26eb6c011e3ab09b4c1e0029dcc7c73411571f0ea1290e088f1101093d not found: ID does not exist"
Jul 09 04:03:31.060344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060313    2577 scope.go:117] "RemoveContainer" containerID="f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4"
Jul 09 04:03:31.060551 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:03:31.060532    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4\": container with ID starting with f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4 not found: ID does not exist" containerID="f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4"
Jul 09 04:03:31.060551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060545    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="alertmanager"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060561    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="alertmanager"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060555    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4"} err="failed to get container status \"f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4\": rpc error: code = NotFound desc = could not find container \"f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4\": container with ID starting with f4e585695008b028ce584713839c1a1bfdb7a941b6d7bcb23ff7e3a5bcc1c1c4 not found: ID does not exist"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060572    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060581    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060591    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-web"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060597    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-web"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060606    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="init-config-reloader"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060612    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="init-config-reloader"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060620    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="prom-label-proxy"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060626    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="prom-label-proxy"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060638    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-metric"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060646    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-metric"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060659    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="config-reloader"
Jul 09 04:03:31.060679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060665    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="config-reloader"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060709    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="alertmanager"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060717    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="config-reloader"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060725    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-web"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060734    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060739    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="kube-rbac-proxy-metric"
Jul 09 04:03:31.061105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.060745    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" containerName="prom-label-proxy"
Jul 09 04:03:31.065611 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.065596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.067378 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067355    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 04:03:31.067452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067420    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jjz8f\""
Jul 09 04:03:31.067513 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 04:03:31.067738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067716    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 04:03:31.067738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 04:03:31.067738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067721    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 04:03:31.067948 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 04:03:31.067948 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.067861    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 04:03:31.068297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.068264    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 04:03:31.072614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.072593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 04:03:31.077400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.077382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:31.096979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.096958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-volume\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.096988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hndgb\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-kube-api-access-hndgb\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-web-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-out\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097424 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.097424 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.097319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.178268 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.178239    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf449ab-0cbe-459c-8cb9-bda5aab7cc45" path="/var/lib/kubelet/pods/6bf449ab-0cbe-459c-8cb9-bda5aab7cc45/volumes"
Jul 09 04:03:31.197752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.197821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-web-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.197874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-out\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.197874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.197947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.197947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.197936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198203    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-volume\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hndgb\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-kube-api-access-hndgb\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.198738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.198692    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.199659 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.199633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.200950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.200925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.200936    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201322    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-web-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-out\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201544 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201521    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201698 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201681    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-config-volume\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.201761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.201688    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.202727 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.202705    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.206363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.206334    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hndgb\" (UniqueName: \"kubernetes.io/projected/12eed2a2-f8a8-4556-98cd-84abfb5d0e32-kube-api-access-hndgb\") pod \"alertmanager-main-0\" (UID: \"12eed2a2-f8a8-4556-98cd-84abfb5d0e32\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.375131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.375092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 04:03:31.514806 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:31.514778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 04:03:31.517617 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:03:31.517591    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12eed2a2_f8a8_4556_98cd_84abfb5d0e32.slice/crio-76aad59e5fbbfec2ef1e9dbbda111c43039f290a53120bd58ab59e374d2526f6 WatchSource:0}: Error finding container 76aad59e5fbbfec2ef1e9dbbda111c43039f290a53120bd58ab59e374d2526f6: Status 404 returned error can't find the container with id 76aad59e5fbbfec2ef1e9dbbda111c43039f290a53120bd58ab59e374d2526f6
Jul 09 04:03:32.015808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:32.015771    2577 generic.go:356] "Generic (PLEG): container finished" podID="12eed2a2-f8a8-4556-98cd-84abfb5d0e32" containerID="c0b6123f1f659afaf14c096597e4930bcfe8852d391d5a9a08f848cc0df9fab2" exitCode=0
Jul 09 04:03:32.016006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:32.015873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerDied","Data":"c0b6123f1f659afaf14c096597e4930bcfe8852d391d5a9a08f848cc0df9fab2"}
Jul 09 04:03:32.016006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:32.015941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"76aad59e5fbbfec2ef1e9dbbda111c43039f290a53120bd58ab59e374d2526f6"}
Jul 09 04:03:33.023655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"3381bde7982377e858b5b555652390ceb1f75b01384d3c53c540b68171cef888"}
Jul 09 04:03:33.024086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"c1c4085d06d926d4caa0bb7112d74b30b25aeac481982e02992165a35bc6318c"}
Jul 09 04:03:33.024086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"113d7c5d5d4ae05acfadb85d9acd03d898c0fe0adf3ab7fb2be40d4716794878"}
Jul 09 04:03:33.024086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"c24cb1d3b633d4e5b61f6fc02da1da722e841e9822ab4fd25e9123d1f2b00839"}
Jul 09 04:03:33.024086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"2c01c0295bb32e7fd3f5568296c207af7b223e28dfd41e669be6cb7e72f6e11f"}
Jul 09 04:03:33.024086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.023827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12eed2a2-f8a8-4556-98cd-84abfb5d0e32","Type":"ContainerStarted","Data":"99cba54c94d7a0120c51d0afd15f7ca9dd45ae0cfa35c5706efbd7159cb87b8d"}
Jul 09 04:03:33.047210 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.047152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.031610412 podStartE2EDuration="2.047130262s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.51985ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:03:33.04557969 +0000 UTC m=+212.319957616" watchObservedRunningTime="2026-07-09 04:03:33.047130262 +0000 UTC m=+212.321508171"
Jul 09 04:03:33.179181 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.179150    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"]
Jul 09 04:03:33.182682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.182662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.185043 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 04:03:33.185176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185051    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 04:03:33.185176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-clkb5\""
Jul 09 04:03:33.185176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 04:03:33.185346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185172    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 04:03:33.185440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.185383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 04:03:33.189979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.189958    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 04:03:33.194941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.194804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"]
Jul 09 04:03:33.213431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-serving-certs-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85t22\" (UniqueName: \"kubernetes.io/projected/875f2d19-c79e-4f9b-8363-139af5d25b0c-kube-api-access-85t22\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-federate-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213729 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213729 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.213795 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.213746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-metrics-client-ca\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-metrics-client-ca\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-serving-certs-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85t22\" (UniqueName: \"kubernetes.io/projected/875f2d19-c79e-4f9b-8363-139af5d25b0c-kube-api-access-85t22\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-federate-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.314502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.314347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.315116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.315088    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-metrics-client-ca\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.315234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.315172    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.315528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.315511    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/875f2d19-c79e-4f9b-8363-139af5d25b0c-serving-certs-ca-bundle\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.316704 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.316668    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.316930 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.316908    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-secret-telemeter-client\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.317164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.317148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-telemeter-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.317205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.317189    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/875f2d19-c79e-4f9b-8363-139af5d25b0c-federate-client-tls\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.321582 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.321556    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85t22\" (UniqueName: \"kubernetes.io/projected/875f2d19-c79e-4f9b-8363-139af5d25b0c-kube-api-access-85t22\") pod \"telemeter-client-7c5d9988d9-9j2td\" (UID: \"875f2d19-c79e-4f9b-8363-139af5d25b0c\") " pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.492678 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.492647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"
Jul 09 04:03:33.624107 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:33.624065    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c5d9988d9-9j2td"]
Jul 09 04:03:33.628247 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:03:33.628222    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875f2d19_c79e_4f9b_8363_139af5d25b0c.slice/crio-1627e6a101446a155da79e37144db1fc2510c9efd7346a8d3522070d9deec5bc WatchSource:0}: Error finding container 1627e6a101446a155da79e37144db1fc2510c9efd7346a8d3522070d9deec5bc: Status 404 returned error can't find the container with id 1627e6a101446a155da79e37144db1fc2510c9efd7346a8d3522070d9deec5bc
Jul 09 04:03:34.028443 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:34.028408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td" event={"ID":"875f2d19-c79e-4f9b-8363-139af5d25b0c","Type":"ContainerStarted","Data":"1627e6a101446a155da79e37144db1fc2510c9efd7346a8d3522070d9deec5bc"}
Jul 09 04:03:37.041986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:37.041949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td" event={"ID":"875f2d19-c79e-4f9b-8363-139af5d25b0c","Type":"ContainerStarted","Data":"f98037b9a114f45dd6ecbfa636d5ba7668dcf90b46d2a1ecb122284d784e7ac6"}
Jul 09 04:03:37.042365 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:37.042005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td" event={"ID":"875f2d19-c79e-4f9b-8363-139af5d25b0c","Type":"ContainerStarted","Data":"578becdbfd92a3a17af5174b75ae36305653887edff39976638b4a1d77c87403"}
Jul 09 04:03:37.042365 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:37.042024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td" event={"ID":"875f2d19-c79e-4f9b-8363-139af5d25b0c","Type":"ContainerStarted","Data":"19bdedecfad34b0d41bf9e2e5996aafae656b2d151dce15af408dc18f46162b0"}
Jul 09 04:03:37.065545 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:37.065496    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c5d9988d9-9j2td" podStartSLOduration=1.516779798 podStartE2EDuration="4.065462265s" totalImagesPullingTime="2.548682467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:03:37.062944982 +0000 UTC m=+216.337322901" watchObservedRunningTime="2026-07-09 04:03:37.065462265 +0000 UTC m=+216.339840173"
Jul 09 04:03:38.058145 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:38.058103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:38.200513 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:38.200480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:38.259678 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:38.259614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:38.787310 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:38.787276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:38.836957 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:38.836919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:49.278800 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:49.278746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:49.331878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:49.331847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:03:49.473743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:03:49.473709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:18.146895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:18.146857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:18.247408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:18.247381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:43.363111 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:43.363072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:43.463490 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:43.463439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:47.566383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:47.566347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:51.163056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:51.163005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:51.232478 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:51.232424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:55.343216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:55.343180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:04:55.444847 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:04:55.444813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:01.037671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:01.037639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:05:01.041079 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:01.041059    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:05:01.049715 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:01.049690    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 04:05:12.127808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:12.127771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:12.226334 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:12.226303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:12.522215 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:12.522142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:16.115936 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:16.115901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:16.217028 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:16.216992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:16.277590 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:16.277559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:17.051278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:17.051244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:17.116506 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:17.116460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:26.653350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:26.653308    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:26.754363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:26.754328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:26.804747 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:26.804713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:45.891661 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:45.891626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:46.515953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.515923    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"]
Jul 09 04:05:46.519392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.519373    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.521177 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.521155    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pft25\""
Jul 09 04:05:46.521234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.521155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 04:05:46.521413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.521399    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 04:05:46.526013 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.525990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"]
Jul 09 04:05:46.588263 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.588220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlbtm\" (UniqueName: \"kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.588454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.588276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.588454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.588353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.689199 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.689159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlbtm\" (UniqueName: \"kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.689373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.689210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.689373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.689252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.689719 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.689699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.689761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.689714    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.697092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.697066    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlbtm\" (UniqueName: \"kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.829245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.829159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:05:46.957748 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.957716    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"]
Jul 09 04:05:46.960646 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:05:46.960621    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482e4d6b_ccac_4e47_b578_c966e030d8a4.slice/crio-9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523 WatchSource:0}: Error finding container 9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523: Status 404 returned error can't find the container with id 9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523
Jul 09 04:05:46.962518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:46.962502    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:05:47.519143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:47.519102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc" event={"ID":"482e4d6b-ccac-4e47-b578-c966e030d8a4","Type":"ContainerStarted","Data":"9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523"}
Jul 09 04:05:47.694314 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:47.694269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:05:55.553936 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:55.553903    2577 generic.go:356] "Generic (PLEG): container finished" podID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerID="185ee7e17b1b238d75a8e4f00369f980b99be02c1fd52bd18c27582011234bdd" exitCode=0
Jul 09 04:05:55.554430 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:55.553975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc" event={"ID":"482e4d6b-ccac-4e47-b578-c966e030d8a4","Type":"ContainerDied","Data":"185ee7e17b1b238d75a8e4f00369f980b99be02c1fd52bd18c27582011234bdd"}
Jul 09 04:05:58.570796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:58.570762    2577 generic.go:356] "Generic (PLEG): container finished" podID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerID="3bd728f445d20f98842e53aa47f2d49dceda39f6f97b8bb5e958fcea3601c226" exitCode=0
Jul 09 04:05:58.571172 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:05:58.570809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc" event={"ID":"482e4d6b-ccac-4e47-b578-c966e030d8a4","Type":"ContainerDied","Data":"3bd728f445d20f98842e53aa47f2d49dceda39f6f97b8bb5e958fcea3601c226"}
Jul 09 04:06:05.602824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:05.602785    2577 generic.go:356] "Generic (PLEG): container finished" podID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerID="65cb3ef041e9aca80ca334a32046a1bee7e3b6fd0a5c4ec678017f741303887f" exitCode=0
Jul 09 04:06:05.602824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:05.602825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc" event={"ID":"482e4d6b-ccac-4e47-b578-c966e030d8a4","Type":"ContainerDied","Data":"65cb3ef041e9aca80ca334a32046a1bee7e3b6fd0a5c4ec678017f741303887f"}
Jul 09 04:06:06.735828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.735802    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:06:06.867680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.867593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlbtm\" (UniqueName: \"kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm\") pod \"482e4d6b-ccac-4e47-b578-c966e030d8a4\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") "
Jul 09 04:06:06.867680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.867646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util\") pod \"482e4d6b-ccac-4e47-b578-c966e030d8a4\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") "
Jul 09 04:06:06.867894 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.867707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle\") pod \"482e4d6b-ccac-4e47-b578-c966e030d8a4\" (UID: \"482e4d6b-ccac-4e47-b578-c966e030d8a4\") "
Jul 09 04:06:06.868387 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.868357    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle" (OuterVolumeSpecName: "bundle") pod "482e4d6b-ccac-4e47-b578-c966e030d8a4" (UID: "482e4d6b-ccac-4e47-b578-c966e030d8a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:06.870008 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.869975    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm" (OuterVolumeSpecName: "kube-api-access-xlbtm") pod "482e4d6b-ccac-4e47-b578-c966e030d8a4" (UID: "482e4d6b-ccac-4e47-b578-c966e030d8a4"). InnerVolumeSpecName "kube-api-access-xlbtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:06:06.871798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.871775    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util" (OuterVolumeSpecName: "util") pod "482e4d6b-ccac-4e47-b578-c966e030d8a4" (UID: "482e4d6b-ccac-4e47-b578-c966e030d8a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:06.968745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.968708    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:06.968745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.968739    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlbtm\" (UniqueName: \"kubernetes.io/projected/482e4d6b-ccac-4e47-b578-c966e030d8a4-kube-api-access-xlbtm\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:06.968745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:06.968748    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/482e4d6b-ccac-4e47-b578-c966e030d8a4-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:07.613030 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:07.612996    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc"
Jul 09 04:06:07.613030 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:07.613007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cgkzc" event={"ID":"482e4d6b-ccac-4e47-b578-c966e030d8a4","Type":"ContainerDied","Data":"9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523"}
Jul 09 04:06:07.613250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:07.613058    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e83466440e7be4a66e50727a10f70beebd208082d8d2137f5714142742bc523"
Jul 09 04:06:12.531520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:12.531463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:13.324183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:13.324149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:16.156695 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:16.156660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:16.256454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:16.256423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:18.883154 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.883121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:18.985709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.985671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:18.989017 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.988995    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bt8tf"]
Jul 09 04:06:18.989387 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989369    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="util"
Jul 09 04:06:18.989494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="util"
Jul 09 04:06:18.989494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989404    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="extract"
Jul 09 04:06:18.989494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989412    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="extract"
Jul 09 04:06:18.989494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989439    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="pull"
Jul 09 04:06:18.989494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="pull"
Jul 09 04:06:18.989660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.989576    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="482e4d6b-ccac-4e47-b578-c966e030d8a4" containerName="extract"
Jul 09 04:06:18.995387 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.995372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:18.997937 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.997914    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 04:06:18.998626 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:18.998609    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 04:06:19.004459 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.004439    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bt8tf"]
Jul 09 04:06:19.063570 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.063537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.063732 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.063586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwhp6\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-kube-api-access-xwhp6\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.134447 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.134347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:19.164178 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.164143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwhp6\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-kube-api-access-xwhp6\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.164329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.164282    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.179144 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.179111    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.179266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.179215    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwhp6\" (UniqueName: \"kubernetes.io/projected/98d6b11d-fda1-4a35-b6ce-a018ee7f74f1-kube-api-access-xwhp6\") pod \"cert-manager-webhook-6486ffd997-bt8tf\" (UID: \"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.238528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.238498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:19.246299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.246270    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"]
Jul 09 04:06:19.249871 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.249855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.252027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.252006    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 04:06:19.252027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.252017    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 04:06:19.252184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.252017    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pft25\""
Jul 09 04:06:19.260877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.260858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"]
Jul 09 04:06:19.265456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.265437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55lp4\" (UniqueName: \"kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.265556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.265488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.265556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.265536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.305595 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.305563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:19.367510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.366804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55lp4\" (UniqueName: \"kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.367510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.366844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.367510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.366871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.367510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.367162    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.367864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.367842    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.376027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.375964    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55lp4\" (UniqueName: \"kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.441595 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.441440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bt8tf"]
Jul 09 04:06:19.444288 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:06:19.444263    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98d6b11d_fda1_4a35_b6ce_a018ee7f74f1.slice/crio-fe39dfa4ae15a74eba213844c323f13f0ac55da04f86985647448cf6818e64c6 WatchSource:0}: Error finding container fe39dfa4ae15a74eba213844c323f13f0ac55da04f86985647448cf6818e64c6: Status 404 returned error can't find the container with id fe39dfa4ae15a74eba213844c323f13f0ac55da04f86985647448cf6818e64c6
Jul 09 04:06:19.558773 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.558737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:19.664533 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.664500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf" event={"ID":"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1","Type":"ContainerStarted","Data":"fe39dfa4ae15a74eba213844c323f13f0ac55da04f86985647448cf6818e64c6"}
Jul 09 04:06:19.685594 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:19.685565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"]
Jul 09 04:06:19.689226 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:06:19.689204    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a95527_7461_4332_b55c_89538edbb5f5.slice/crio-83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b WatchSource:0}: Error finding container 83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b: Status 404 returned error can't find the container with id 83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b
Jul 09 04:06:20.671740 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:20.671705    2577 generic.go:356] "Generic (PLEG): container finished" podID="36a95527-7461-4332-b55c-89538edbb5f5" containerID="caa55a10dbdff3fa0e6f61fdba1dfa170c4caa04ee54224615d4e8ac02f90d58" exitCode=0
Jul 09 04:06:20.672223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:20.671805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5" event={"ID":"36a95527-7461-4332-b55c-89538edbb5f5","Type":"ContainerDied","Data":"caa55a10dbdff3fa0e6f61fdba1dfa170c4caa04ee54224615d4e8ac02f90d58"}
Jul 09 04:06:20.672223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:20.671866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5" event={"ID":"36a95527-7461-4332-b55c-89538edbb5f5","Type":"ContainerStarted","Data":"83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b"}
Jul 09 04:06:20.859666 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:20.859631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:21.257163 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:21.257133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:22.944290 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:22.944252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:23.045176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.045139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:23.688517 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.688483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf" event={"ID":"98d6b11d-fda1-4a35-b6ce-a018ee7f74f1","Type":"ContainerStarted","Data":"b6743b1f21634ed80c8b40aaaeb57dfab2e21c89100ce269005fbd5143fc5894"}
Jul 09 04:06:23.688704 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.688526    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:23.690086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.690062    2577 generic.go:356] "Generic (PLEG): container finished" podID="36a95527-7461-4332-b55c-89538edbb5f5" containerID="7673597779e8c87295bd6170f41e6304963b9abe182513acf58f42ea78075328" exitCode=0
Jul 09 04:06:23.690194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.690097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5" event={"ID":"36a95527-7461-4332-b55c-89538edbb5f5","Type":"ContainerDied","Data":"7673597779e8c87295bd6170f41e6304963b9abe182513acf58f42ea78075328"}
Jul 09 04:06:23.735626 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:23.735581    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf" podStartSLOduration=2.207093 podStartE2EDuration="5.735562916s" totalImagesPullingTime="3.528469916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:06:23.70867006 +0000 UTC m=+382.983047969" watchObservedRunningTime="2026-07-09 04:06:23.735562916 +0000 UTC m=+383.009940824"
Jul 09 04:06:24.695913 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:24.695876    2577 generic.go:356] "Generic (PLEG): container finished" podID="36a95527-7461-4332-b55c-89538edbb5f5" containerID="0016ef05259983e8debb8f156ce2a44edf28c1d46942934d860bcdb2d24842eb" exitCode=0
Jul 09 04:06:24.696345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:24.695950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5" event={"ID":"36a95527-7461-4332-b55c-89538edbb5f5","Type":"ContainerDied","Data":"0016ef05259983e8debb8f156ce2a44edf28c1d46942934d860bcdb2d24842eb"}
Jul 09 04:06:25.822418 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.822394    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:25.924679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.924642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55lp4\" (UniqueName: \"kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4\") pod \"36a95527-7461-4332-b55c-89538edbb5f5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") "
Jul 09 04:06:25.924846 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.924705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle\") pod \"36a95527-7461-4332-b55c-89538edbb5f5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") "
Jul 09 04:06:25.924846 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.924741    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util\") pod \"36a95527-7461-4332-b55c-89538edbb5f5\" (UID: \"36a95527-7461-4332-b55c-89538edbb5f5\") "
Jul 09 04:06:25.925142 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.925118    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle" (OuterVolumeSpecName: "bundle") pod "36a95527-7461-4332-b55c-89538edbb5f5" (UID: "36a95527-7461-4332-b55c-89538edbb5f5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:25.926864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.926833    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4" (OuterVolumeSpecName: "kube-api-access-55lp4") pod "36a95527-7461-4332-b55c-89538edbb5f5" (UID: "36a95527-7461-4332-b55c-89538edbb5f5"). InnerVolumeSpecName "kube-api-access-55lp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:06:25.928903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:25.928884    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util" (OuterVolumeSpecName: "util") pod "36a95527-7461-4332-b55c-89538edbb5f5" (UID: "36a95527-7461-4332-b55c-89538edbb5f5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:26.026116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.026033    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55lp4\" (UniqueName: \"kubernetes.io/projected/36a95527-7461-4332-b55c-89538edbb5f5-kube-api-access-55lp4\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:26.026116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.026061    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:26.026116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.026069    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a95527-7461-4332-b55c-89538edbb5f5-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:26.705875 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.705849    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5"
Jul 09 04:06:26.705875 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.705871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frz4b5" event={"ID":"36a95527-7461-4332-b55c-89538edbb5f5","Type":"ContainerDied","Data":"83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b"}
Jul 09 04:06:26.706076 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:26.705914    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83a535a45aa174bebc55eb3b9d4161cb0d1e40e4ca192e6b03062c2ed9af9f5b"
Jul 09 04:06:29.472854 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:29.472819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:29.697843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:29.697816    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bt8tf"
Jul 09 04:06:30.250644 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:30.250615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:30.350406 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:30.350369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:30.864955 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:30.864919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:30.924574 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:30.924535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:37.532718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:37.532680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:37.630572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:37.630535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:39.319797 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.319765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:39.458903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.458869    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"]
Jul 09 04:06:39.459192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="util"
Jul 09 04:06:39.459234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459193    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="util"
Jul 09 04:06:39.459234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459202    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="pull"
Jul 09 04:06:39.459234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459208    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="pull"
Jul 09 04:06:39.459234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="extract"
Jul 09 04:06:39.459234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459230    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="extract"
Jul 09 04:06:39.459381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.459279    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a95527-7461-4332-b55c-89538edbb5f5" containerName="extract"
Jul 09 04:06:39.463300 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.463279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.465328 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.465310    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 04:06:39.465702 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.465683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 04:06:39.465802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.465693    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pft25\""
Jul 09 04:06:39.470131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.470111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"]
Jul 09 04:06:39.545325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.545286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.545543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.545387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.545543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.545414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fj52\" (UniqueName: \"kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.574623 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.574554    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nfs8n"]
Jul 09 04:06:39.577562 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.577546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.579290 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.579274    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jvvc7\""
Jul 09 04:06:39.586615 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.586595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nfs8n"]
Jul 09 04:06:39.646408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.646408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fj52\" (UniqueName: \"kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.646689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.646689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-bound-sa-token\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.646689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hl2\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-kube-api-access-67hl2\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.646868 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646851    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.646907 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.646866    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.669622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.669590    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fj52\" (UniqueName: \"kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.747858 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.747826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hl2\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-kube-api-access-67hl2\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.748020 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.747887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-bound-sa-token\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.755837 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.755806    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-bound-sa-token\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.755963 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.755886    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hl2\" (UniqueName: \"kubernetes.io/projected/157b036b-10b3-40e5-b0f4-9ff0ca2a6455-kube-api-access-67hl2\") pod \"cert-manager-55b4d954d5-nfs8n\" (UID: \"157b036b-10b3-40e5-b0f4-9ff0ca2a6455\") " pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.773814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.773791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:39.886840 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.886809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nfs8n"
Jul 09 04:06:39.902259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:39.902241    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"]
Jul 09 04:06:39.904845 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:06:39.904817    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9577543_d214_477f_8871_cfc345584412.slice/crio-a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32 WatchSource:0}: Error finding container a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32: Status 404 returned error can't find the container with id a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32
Jul 09 04:06:40.044921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.044899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nfs8n"]
Jul 09 04:06:40.059281 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:06:40.059253    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod157b036b_10b3_40e5_b0f4_9ff0ca2a6455.slice/crio-7fa04c0995d212d6dce0f70327403a97e2f7198d4bcddec50c551abc02115e55 WatchSource:0}: Error finding container 7fa04c0995d212d6dce0f70327403a97e2f7198d4bcddec50c551abc02115e55: Status 404 returned error can't find the container with id 7fa04c0995d212d6dce0f70327403a97e2f7198d4bcddec50c551abc02115e55
Jul 09 04:06:40.762020 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.761986    2577 generic.go:356] "Generic (PLEG): container finished" podID="e9577543-d214-477f-8871-cfc345584412" containerID="90b4a6a05fdfe9ad7834aec6f87a2280396d890bea730ca8ff73bd757c06a36a" exitCode=0
Jul 09 04:06:40.762529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.762086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g" event={"ID":"e9577543-d214-477f-8871-cfc345584412","Type":"ContainerDied","Data":"90b4a6a05fdfe9ad7834aec6f87a2280396d890bea730ca8ff73bd757c06a36a"}
Jul 09 04:06:40.762529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.762173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g" event={"ID":"e9577543-d214-477f-8871-cfc345584412","Type":"ContainerStarted","Data":"a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32"}
Jul 09 04:06:40.763724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.763704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nfs8n" event={"ID":"157b036b-10b3-40e5-b0f4-9ff0ca2a6455","Type":"ContainerStarted","Data":"e7acf0ec91cb9495f48d9e79c3e076844d2d541b707d227220f3f86144853827"}
Jul 09 04:06:40.763808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.763734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nfs8n" event={"ID":"157b036b-10b3-40e5-b0f4-9ff0ca2a6455","Type":"ContainerStarted","Data":"7fa04c0995d212d6dce0f70327403a97e2f7198d4bcddec50c551abc02115e55"}
Jul 09 04:06:40.793135 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:40.793088    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nfs8n" podStartSLOduration=1.793073635 podStartE2EDuration="1.793073635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:06:40.791949889 +0000 UTC m=+400.066327824" watchObservedRunningTime="2026-07-09 04:06:40.793073635 +0000 UTC m=+400.067451544"
Jul 09 04:06:41.769796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:41.769707    2577 generic.go:356] "Generic (PLEG): container finished" podID="e9577543-d214-477f-8871-cfc345584412" containerID="0833af0756940de568d3983caadb1eafdf3d05c7c7c3081829e0922a4f4e3063" exitCode=0
Jul 09 04:06:41.770216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:41.769795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g" event={"ID":"e9577543-d214-477f-8871-cfc345584412","Type":"ContainerDied","Data":"0833af0756940de568d3983caadb1eafdf3d05c7c7c3081829e0922a4f4e3063"}
Jul 09 04:06:42.776741 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:42.776710    2577 generic.go:356] "Generic (PLEG): container finished" podID="e9577543-d214-477f-8871-cfc345584412" containerID="e69974c9b579997142d25f931104907c5a4e544c338c0989e3313960ba1fd7ca" exitCode=0
Jul 09 04:06:42.777122 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:42.776753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g" event={"ID":"e9577543-d214-477f-8871-cfc345584412","Type":"ContainerDied","Data":"e69974c9b579997142d25f931104907c5a4e544c338c0989e3313960ba1fd7ca"}
Jul 09 04:06:43.904919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.904895    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:43.984225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.984193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fj52\" (UniqueName: \"kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52\") pod \"e9577543-d214-477f-8871-cfc345584412\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") "
Jul 09 04:06:43.984411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.984301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util\") pod \"e9577543-d214-477f-8871-cfc345584412\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") "
Jul 09 04:06:43.984411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.984328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle\") pod \"e9577543-d214-477f-8871-cfc345584412\" (UID: \"e9577543-d214-477f-8871-cfc345584412\") "
Jul 09 04:06:43.985087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.985062    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle" (OuterVolumeSpecName: "bundle") pod "e9577543-d214-477f-8871-cfc345584412" (UID: "e9577543-d214-477f-8871-cfc345584412"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:43.986259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.986238    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52" (OuterVolumeSpecName: "kube-api-access-4fj52") pod "e9577543-d214-477f-8871-cfc345584412" (UID: "e9577543-d214-477f-8871-cfc345584412"). InnerVolumeSpecName "kube-api-access-4fj52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:06:43.993248 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:43.993151    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util" (OuterVolumeSpecName: "util") pod "e9577543-d214-477f-8871-cfc345584412" (UID: "e9577543-d214-477f-8871-cfc345584412"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:06:44.085986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.085903    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fj52\" (UniqueName: \"kubernetes.io/projected/e9577543-d214-477f-8871-cfc345584412-kube-api-access-4fj52\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:44.085986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.085941    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:44.085986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.085955    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9577543-d214-477f-8871-cfc345584412-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:06:44.786923 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.786891    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g" event={"ID":"e9577543-d214-477f-8871-cfc345584412","Type":"ContainerDied","Data":"a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32"}
Jul 09 04:06:44.787063 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.786929    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5fz8g"
Jul 09 04:06:44.787063 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:44.786934    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0d4980cba0954a878bdaea3bdb91b6ff23b2735b240b741f017417cbe377b32"
Jul 09 04:06:55.183112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:55.183077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:55.276266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:55.276233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:55.801356 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:55.801307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.061556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.061450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.171170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.171109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.210823 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.210790    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"]
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211114    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="extract"
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211126    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="extract"
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211136    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="util"
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211144    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="util"
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211162    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="pull"
Jul 09 04:06:56.211174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211168    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="pull"
Jul 09 04:06:56.211360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.211213    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9577543-d214-477f-8871-cfc345584412" containerName="extract"
Jul 09 04:06:56.214301 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.214274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.216668 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.216646    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 04:06:56.216832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.216814    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 04:06:56.216886 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.216871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7rq67\""
Jul 09 04:06:56.217052 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.217038    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 04:06:56.217280 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.217263    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 04:06:56.243084 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.243055    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"]
Jul 09 04:06:56.389912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.389878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2wv2\" (UniqueName: \"kubernetes.io/projected/f171cc86-f9e1-4760-928b-04d849fcdaa3-kube-api-access-t2wv2\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.390083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.389940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.390083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.389979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.390083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.389998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.390083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.390043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.392964 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.392941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.490730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.490692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.490903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.490743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.490903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.490763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.490903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.490797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2wv2\" (UniqueName: \"kubernetes.io/projected/f171cc86-f9e1-4760-928b-04d849fcdaa3-kube-api-access-t2wv2\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.490903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.490860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.491285 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.491251    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.492451 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.492420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.493162 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.493142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f171cc86-f9e1-4760-928b-04d849fcdaa3-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.493541 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.493524    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.493618 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.493566    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f171cc86-f9e1-4760-928b-04d849fcdaa3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.498820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.498800    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2wv2\" (UniqueName: \"kubernetes.io/projected/f171cc86-f9e1-4760-928b-04d849fcdaa3-kube-api-access-t2wv2\") pod \"opendatahub-operator-controller-manager-df844f965-8zh6m\" (UID: \"f171cc86-f9e1-4760-928b-04d849fcdaa3\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.524442 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.524422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:06:56.593231 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.593188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.679260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.679239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"]
Jul 09 04:06:56.681192 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:06:56.681167    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf171cc86_f9e1_4760_928b_04d849fcdaa3.slice/crio-925be31fde932e5dc9db785aecc78a265d70cb746cab5fd8daec816e58a2f8d1 WatchSource:0}: Error finding container 925be31fde932e5dc9db785aecc78a265d70cb746cab5fd8daec816e58a2f8d1: Status 404 returned error can't find the container with id 925be31fde932e5dc9db785aecc78a265d70cb746cab5fd8daec816e58a2f8d1
Jul 09 04:06:56.693663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.693640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.835624 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.835591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m" event={"ID":"f171cc86-f9e1-4760-928b-04d849fcdaa3","Type":"ContainerStarted","Data":"925be31fde932e5dc9db785aecc78a265d70cb746cab5fd8daec816e58a2f8d1"}
Jul 09 04:06:56.847887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.847860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:56.931887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:56.931804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:57.030807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:57.030759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:59.622495 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:59.622431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:59.720165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:59.719904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:59.853455 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:06:59.853427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m" event={"ID":"f171cc86-f9e1-4760-928b-04d849fcdaa3","Type":"ContainerStarted","Data":"1c4b6ef84fde8b8a8856479c166e7360046a55919f82f511bce65993f401e9ee"}
Jul 09 04:07:00.859133 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:00.859094    2577 generic.go:356] "Generic (PLEG): container finished" podID="f171cc86-f9e1-4760-928b-04d849fcdaa3" containerID="1c4b6ef84fde8b8a8856479c166e7360046a55919f82f511bce65993f401e9ee" exitCode=0
Jul 09 04:07:00.859443 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:00.859179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m" event={"ID":"f171cc86-f9e1-4760-928b-04d849fcdaa3","Type":"ContainerDied","Data":"1c4b6ef84fde8b8a8856479c166e7360046a55919f82f511bce65993f401e9ee"}
Jul 09 04:07:01.868774 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:01.868733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m" event={"ID":"f171cc86-f9e1-4760-928b-04d849fcdaa3","Type":"ContainerStarted","Data":"d248ce2995bd92e2d63e9f174ed1a404e4687053a2a54ec981c1554875094256"}
Jul 09 04:07:01.869576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:01.869548    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:07:01.893853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:01.893800    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m" podStartSLOduration=1.35381842 podStartE2EDuration="5.893783158s" totalImagesPullingTime="3.502854345s" totalInitContainerRuntime="1.037110393s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:01.892439244 +0000 UTC m=+421.166817176" watchObservedRunningTime="2026-07-09 04:07:01.893783158 +0000 UTC m=+421.168161065"
Jul 09 04:07:08.778205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:08.778165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:08.869438 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:08.869368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:13.879695 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:13.879662    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-8zh6m"
Jul 09 04:07:15.430717 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:15.430682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:15.542377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:15.542347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.129725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.129691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.226519 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.226455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.326071 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.326030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.627615 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.627582    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f25h2"]
Jul 09 04:07:16.633391 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.633364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.641616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.641594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.642403 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.642382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f25h2"]
Jul 09 04:07:16.643835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.643816    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 09 04:07:16.755952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.755911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzr8\" (UniqueName: \"kubernetes.io/projected/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-kube-api-access-lhzr8\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.756134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.755958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-tls-certs\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.770642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.770614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:16.857014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.856976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzr8\" (UniqueName: \"kubernetes.io/projected/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-kube-api-access-lhzr8\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.857660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.857634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-tls-certs\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.860768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.860742    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-tls-certs\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.872937 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.872914    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzr8\" (UniqueName: \"kubernetes.io/projected/2fbf5964-e416-4540-a6fa-dc30ab7f24f4-kube-api-access-lhzr8\") pod \"model-serving-api-54d89bd5fc-f25h2\" (UID: \"2fbf5964-e416-4540-a6fa-dc30ab7f24f4\") " pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:16.956461 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:16.956361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:17.309206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:17.309182    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f25h2"]
Jul 09 04:07:17.311390 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:07:17.311361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbf5964_e416_4540_a6fa_dc30ab7f24f4.slice/crio-fb58495621034e7a6cc867478cb584a84e352b859248e8450ee6c346b42d134d WatchSource:0}: Error finding container fb58495621034e7a6cc867478cb584a84e352b859248e8450ee6c346b42d134d: Status 404 returned error can't find the container with id fb58495621034e7a6cc867478cb584a84e352b859248e8450ee6c346b42d134d
Jul 09 04:07:17.940816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:17.940782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2" event={"ID":"2fbf5964-e416-4540-a6fa-dc30ab7f24f4","Type":"ContainerStarted","Data":"fb58495621034e7a6cc867478cb584a84e352b859248e8450ee6c346b42d134d"}
Jul 09 04:07:18.486748 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.486700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:18.511245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.511217    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"]
Jul 09 04:07:18.514559 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.514538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.516837 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.516805    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 04:07:18.516837 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.516816    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 04:07:18.517046 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.516808    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 04:07:18.517092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.517075    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 04:07:18.522809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.522790    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"]
Jul 09 04:07:18.570917 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.570890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.571083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.570944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20f62427-52fa-4201-ad10-ed3adf3e8dea-tmp\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.571083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.571058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shd8\" (UniqueName: \"kubernetes.io/projected/20f62427-52fa-4201-ad10-ed3adf3e8dea-kube-api-access-5shd8\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.672502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.672456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.672691 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.672554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20f62427-52fa-4201-ad10-ed3adf3e8dea-tmp\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.672691 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:18.672614    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 09 04:07:18.672812 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:18.672696    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs podName:20f62427-52fa-4201-ad10-ed3adf3e8dea nodeName:}" failed. No retries permitted until 2026-07-09 04:07:19.17267523 +0000 UTC m=+438.447053121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs") pod "kube-auth-proxy-779b7dc9c4-hl5dr" (UID: "20f62427-52fa-4201-ad10-ed3adf3e8dea") : secret "kube-auth-proxy-tls" not found
Jul 09 04:07:18.672812 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.672613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shd8\" (UniqueName: \"kubernetes.io/projected/20f62427-52fa-4201-ad10-ed3adf3e8dea-kube-api-access-5shd8\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.675092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.675059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20f62427-52fa-4201-ad10-ed3adf3e8dea-tmp\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.681669 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.681641    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shd8\" (UniqueName: \"kubernetes.io/projected/20f62427-52fa-4201-ad10-ed3adf3e8dea-kube-api-access-5shd8\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:18.721941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:18.721905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.177627 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.177595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:19.180275 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.180252    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f62427-52fa-4201-ad10-ed3adf3e8dea-tls-certs\") pod \"kube-auth-proxy-779b7dc9c4-hl5dr\" (UID: \"20f62427-52fa-4201-ad10-ed3adf3e8dea\") " pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:19.424343 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.424311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"
Jul 09 04:07:19.578948 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.578644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr"]
Jul 09 04:07:19.580917 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:07:19.580887    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f62427_52fa_4201_ad10_ed3adf3e8dea.slice/crio-6816ee0705c321550140cb4ed0363f5750a187415822ae0a745921d27085817a WatchSource:0}: Error finding container 6816ee0705c321550140cb4ed0363f5750a187415822ae0a745921d27085817a: Status 404 returned error can't find the container with id 6816ee0705c321550140cb4ed0363f5750a187415822ae0a745921d27085817a
Jul 09 04:07:19.613683 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.613657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.617284 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.617261    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:19.620744 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.620730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.622495 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.622478    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 04:07:19.622625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.622479    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 04:07:19.622787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.622774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 04:07:19.624194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.624174    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 04:07:19.624194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.624191    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xrr7n\""
Jul 09 04:07:19.634919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.634899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:19.681885 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.681852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wps76\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682055 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.681901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682055 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.681996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.682048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.682076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.682127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.682306 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.682195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.696452 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.696414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.782836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.782836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.782836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wps76\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.783124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.783124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.783124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.783124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.782944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.783585 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.783559    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.785225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.785195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.785320 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.785201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.785556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.785538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.785627 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.785581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.790043 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.790024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.790322 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.790305    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wps76\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76\") pod \"istiod-openshift-gateway-7574459656-s5xbl\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.929890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.929855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:19.954846 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.954795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2" event={"ID":"2fbf5964-e416-4540-a6fa-dc30ab7f24f4","Type":"ContainerStarted","Data":"cf2a7b57d35f4da378fa8af47968d0c48efd3c86a71563e44c232fb2c01f82ec"}
Jul 09 04:07:19.955005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.954897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:19.956743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.956712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr" event={"ID":"20f62427-52fa-4201-ad10-ed3adf3e8dea","Type":"ContainerStarted","Data":"6816ee0705c321550140cb4ed0363f5750a187415822ae0a745921d27085817a"}
Jul 09 04:07:19.970790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:19.970737    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2" podStartSLOduration=1.831993368 podStartE2EDuration="3.970709787s" totalImagesPullingTime="2.138716419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:19.969574588 +0000 UTC m=+439.243952500" watchObservedRunningTime="2026-07-09 04:07:19.970709787 +0000 UTC m=+439.245087696"
Jul 09 04:07:20.071496 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:20.071387    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:20.075803 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:07:20.075777    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e82a69_a681_44e2_ac54_8fb225295283.slice/crio-d14a54c1e372c518a655941880ff30d3e5be2fd9492c2fbc112051dc8d2d6dbd WatchSource:0}: Error finding container d14a54c1e372c518a655941880ff30d3e5be2fd9492c2fbc112051dc8d2d6dbd: Status 404 returned error can't find the container with id d14a54c1e372c518a655941880ff30d3e5be2fd9492c2fbc112051dc8d2d6dbd
Jul 09 04:07:20.726903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:20.726842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:20.973911 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:20.973780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" event={"ID":"28e82a69-a681-44e2-ac54-8fb225295283","Type":"ContainerStarted","Data":"d14a54c1e372c518a655941880ff30d3e5be2fd9492c2fbc112051dc8d2d6dbd"}
Jul 09 04:07:23.106433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:23.106397    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:07:23.106737 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:23.106490    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:07:23.993784 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:23.993732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" event={"ID":"28e82a69-a681-44e2-ac54-8fb225295283","Type":"ContainerStarted","Data":"63a08745530b352f69c8bbf2e616a7f543b8b868612c7e4e73ede1f5a520b985"}
Jul 09 04:07:23.994329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:23.994305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:23.996145 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:23.996113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr" event={"ID":"20f62427-52fa-4201-ad10-ed3adf3e8dea","Type":"ContainerStarted","Data":"21ba6a79d8ff073b1db5286ce6caaaacb1717c947204862ee5b19f26587419c7"}
Jul 09 04:07:24.019031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.018973    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" podStartSLOduration=1.990703088 podStartE2EDuration="5.01895372s" totalImagesPullingTime="3.028250632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:24.014746142 +0000 UTC m=+443.289124049" watchObservedRunningTime="2026-07-09 04:07:24.01895372 +0000 UTC m=+443.293331629"
Jul 09 04:07:24.023428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.023393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:24.045539 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.045460    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779b7dc9c4-hl5dr" podStartSLOduration=2.52358552 podStartE2EDuration="6.045446753s" totalImagesPullingTime="3.521861233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:24.044240555 +0000 UTC m=+443.318618463" watchObservedRunningTime="2026-07-09 04:07:24.045446753 +0000 UTC m=+443.319824660"
Jul 09 04:07:24.099152 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.099117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:24.358013 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.357980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:24.571537 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.571501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"]
Jul 09 04:07:24.574675 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.574659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.576628 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.576612    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qkk4c\""
Jul 09 04:07:24.590318 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.590298    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"]
Jul 09 04:07:24.594864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.594819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:24.633400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtst\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-kube-api-access-4xtst\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.633882 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.633708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtst\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-kube-api-access-4xtst\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734734 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734668    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734717    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734897 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734875    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.734972 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.734950    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.735096 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.735073    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.736663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.736639    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.737014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.736993    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.756283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.756246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.757021 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.757000    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtst\" (UniqueName: \"kubernetes.io/projected/5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8-kube-api-access-4xtst\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787l6q4\" (UID: \"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:24.885138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:24.885049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:25.004839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:25.004814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:25.019268 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:25.019237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"]
Jul 09 04:07:25.399939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:25.399908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:26.010065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:26.010025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4" event={"ID":"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8","Type":"ContainerStarted","Data":"decca6191f9e311e9642b2e30ee1b0a5bc091a4a4f46e28321005c945de355b7"}
Jul 09 04:07:27.016013 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.015830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.070877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.069093    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"]
Jul 09 04:07:27.076627 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.076600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.078702 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.078678    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 09 04:07:27.078832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.078779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jwcvh\""
Jul 09 04:07:27.083309 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.083289    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"]
Jul 09 04:07:27.087319 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.087302    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.089684 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.089660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"]
Jul 09 04:07:27.092385 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.092319    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9zgtl\""
Jul 09 04:07:27.092571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.092551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 09 04:07:27.105851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.105829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"]
Jul 09 04:07:27.109717 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.109690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.160896 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.160851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m2x2\" (UniqueName: \"kubernetes.io/projected/1bc717cc-5e30-4221-b74f-89b63b207308-kube-api-access-4m2x2\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.161050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.160916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.161050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.160954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.161050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.160979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbjs\" (UniqueName: \"kubernetes.io/projected/61b9ad92-8594-477c-a5a6-708c2b6904fe-kube-api-access-5jbjs\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.262403 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.262368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m2x2\" (UniqueName: \"kubernetes.io/projected/1bc717cc-5e30-4221-b74f-89b63b207308-kube-api-access-4m2x2\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.262403 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.262413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.262663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.262433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.262663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.262460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbjs\" (UniqueName: \"kubernetes.io/projected/61b9ad92-8594-477c-a5a6-708c2b6904fe-kube-api-access-5jbjs\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.262663 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:27.262558    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 09 04:07:27.262663 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:27.262633    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert podName:1bc717cc-5e30-4221-b74f-89b63b207308 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.762611907 +0000 UTC m=+447.036989806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert") pod "kserve-controller-manager-d6f4b65b8-th7dt" (UID: "1bc717cc-5e30-4221-b74f-89b63b207308") : secret "kserve-webhook-server-cert" not found
Jul 09 04:07:27.262663 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:27.262656    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 09 04:07:27.262889 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:07:27.262715    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert podName:61b9ad92-8594-477c-a5a6-708c2b6904fe nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.762697137 +0000 UTC m=+447.037075027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert") pod "llmisvc-controller-manager-777c6bd7d5-l882p" (UID: "61b9ad92-8594-477c-a5a6-708c2b6904fe") : secret "llmisvc-webhook-server-cert" not found
Jul 09 04:07:27.280750 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.280662    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbjs\" (UniqueName: \"kubernetes.io/projected/61b9ad92-8594-477c-a5a6-708c2b6904fe-kube-api-access-5jbjs\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.280750 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.280696    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m2x2\" (UniqueName: \"kubernetes.io/projected/1bc717cc-5e30-4221-b74f-89b63b207308-kube-api-access-4m2x2\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.417543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.417506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.516670 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.516631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.581564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.581530    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:07:27.581635 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.581605    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:07:27.581669 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.581638    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:07:27.766902 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.766853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.766902 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.766895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.769523 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.769460    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b9ad92-8594-477c-a5a6-708c2b6904fe-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l882p\" (UID: \"61b9ad92-8594-477c-a5a6-708c2b6904fe\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:27.770319 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.770292    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bc717cc-5e30-4221-b74f-89b63b207308-cert\") pod \"kserve-controller-manager-d6f4b65b8-th7dt\" (UID: \"1bc717cc-5e30-4221-b74f-89b63b207308\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.990805 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.990772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:27.999950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:27.999926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:28.024132 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.024098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4" event={"ID":"5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8","Type":"ContainerStarted","Data":"fa48fbf201a9a0ca8eee7dd5a85e3c17f3b922963387c9e6207135dc9d280a74"}
Jul 09 04:07:28.050691 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.050629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4" podStartSLOduration=1.493912965 podStartE2EDuration="4.050604013s" totalImagesPullingTime="2.556691048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:28.048959863 +0000 UTC m=+447.323337772" watchObservedRunningTime="2026-07-09 04:07:28.050604013 +0000 UTC m=+447.324981921"
Jul 09 04:07:28.169491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.169446    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"]
Jul 09 04:07:28.171454 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:07:28.171431    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc717cc_5e30_4221_b74f_89b63b207308.slice/crio-62f3e775581b82b9d12a9c78dda7c701798ab7e2c35f96f0ad8eea812a93eba9 WatchSource:0}: Error finding container 62f3e775581b82b9d12a9c78dda7c701798ab7e2c35f96f0ad8eea812a93eba9: Status 404 returned error can't find the container with id 62f3e775581b82b9d12a9c78dda7c701798ab7e2c35f96f0ad8eea812a93eba9
Jul 09 04:07:28.176544 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.176522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"]
Jul 09 04:07:28.179023 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:07:28.179001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61b9ad92_8594_477c_a5a6_708c2b6904fe.slice/crio-45fb637b8792c8321e14172593a4ddc3662b06f95121145848e69a7f7f995fef WatchSource:0}: Error finding container 45fb637b8792c8321e14172593a4ddc3662b06f95121145848e69a7f7f995fef: Status 404 returned error can't find the container with id 45fb637b8792c8321e14172593a4ddc3662b06f95121145848e69a7f7f995fef
Jul 09 04:07:28.300938 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.300865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:28.400963 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.400928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:28.501686 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.501658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:28.886985 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.886932    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:28.893628 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:28.893598    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:29.031016 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:29.030974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p" event={"ID":"61b9ad92-8594-477c-a5a6-708c2b6904fe","Type":"ContainerStarted","Data":"45fb637b8792c8321e14172593a4ddc3662b06f95121145848e69a7f7f995fef"}
Jul 09 04:07:29.032729 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:29.032698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt" event={"ID":"1bc717cc-5e30-4221-b74f-89b63b207308","Type":"ContainerStarted","Data":"62f3e775581b82b9d12a9c78dda7c701798ab7e2c35f96f0ad8eea812a93eba9"}
Jul 09 04:07:29.032990 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:29.032968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:29.034052 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:29.034033    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787l6q4"
Jul 09 04:07:30.457269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:30.457229    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:30.457798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:30.457448    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" podUID="28e82a69-a681-44e2-ac54-8fb225295283" containerName="discovery" containerID="cri-o://63a08745530b352f69c8bbf2e616a7f543b8b868612c7e4e73ede1f5a520b985" gracePeriod=30
Jul 09 04:07:30.978809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:30.978779    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-f25h2"
Jul 09 04:07:31.045810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.045719    2577 generic.go:356] "Generic (PLEG): container finished" podID="28e82a69-a681-44e2-ac54-8fb225295283" containerID="63a08745530b352f69c8bbf2e616a7f543b8b868612c7e4e73ede1f5a520b985" exitCode=0
Jul 09 04:07:31.045961 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.045800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" event={"ID":"28e82a69-a681-44e2-ac54-8fb225295283","Type":"ContainerDied","Data":"63a08745530b352f69c8bbf2e616a7f543b8b868612c7e4e73ede1f5a520b985"}
Jul 09 04:07:31.301115 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.301062    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:31.407862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407838    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.407956 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.407956 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.407956 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.408106 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wps76\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.408106 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.407996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.408413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.408263    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig\") pod \"28e82a69-a681-44e2-ac54-8fb225295283\" (UID: \"28e82a69-a681-44e2-ac54-8fb225295283\") "
Jul 09 04:07:31.408413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.408380    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:07:31.408629 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.408609    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-ca-configmap\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.410758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.410707    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token" (OuterVolumeSpecName: "istio-token") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:07:31.410758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.410724    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs" (OuterVolumeSpecName: "local-certs") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:07:31.410933 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.410759    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts" (OuterVolumeSpecName: "cacerts") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:31.410933 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.410792    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76" (OuterVolumeSpecName: "kube-api-access-wps76") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "kube-api-access-wps76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:07:31.410933 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.410772    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:31.411821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.411800    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "28e82a69-a681-44e2-ac54-8fb225295283" (UID: "28e82a69-a681-44e2-ac54-8fb225295283"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:31.509027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.508993    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-csr-dns-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.509027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.509021    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wps76\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-kube-api-access-wps76\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.509027 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.509032    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e82a69-a681-44e2-ac54-8fb225295283-local-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.509441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.509042    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-istio-kubeconfig\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.509441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.509049    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28e82a69-a681-44e2-ac54-8fb225295283-cacerts\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:31.509441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:31.509057    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e82a69-a681-44e2-ac54-8fb225295283-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:07:32.051956 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.051862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt" event={"ID":"1bc717cc-5e30-4221-b74f-89b63b207308","Type":"ContainerStarted","Data":"e92aba732e3171cd4dddc7d361f9bf1581c6d553c2eca827ea66458b196adc46"}
Jul 09 04:07:32.052105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.051970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:07:32.053456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.053434    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p" event={"ID":"61b9ad92-8594-477c-a5a6-708c2b6904fe","Type":"ContainerStarted","Data":"7d197babdcd58b681e327418ec6342e264223f39178279fc101a9fc298db8468"}
Jul 09 04:07:32.053598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.053524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:07:32.054843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.054821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" event={"ID":"28e82a69-a681-44e2-ac54-8fb225295283","Type":"ContainerDied","Data":"d14a54c1e372c518a655941880ff30d3e5be2fd9492c2fbc112051dc8d2d6dbd"}
Jul 09 04:07:32.054940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.054861    2577 scope.go:117] "RemoveContainer" containerID="63a08745530b352f69c8bbf2e616a7f543b8b868612c7e4e73ede1f5a520b985"
Jul 09 04:07:32.054940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.054831    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"
Jul 09 04:07:32.066247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.066175    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt" podStartSLOduration=1.92463501 podStartE2EDuration="5.066157713s" totalImagesPullingTime="3.141522703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:32.065912188 +0000 UTC m=+451.340290095" watchObservedRunningTime="2026-07-09 04:07:32.066157713 +0000 UTC m=+451.340535623"
Jul 09 04:07:32.080229 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.080209    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:32.084073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.084051    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl"]
Jul 09 04:07:32.097831 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:32.097791    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p" podStartSLOduration=1.959740372 podStartE2EDuration="5.097777076s" totalImagesPullingTime="3.138036704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:32.096100325 +0000 UTC m=+451.370478254" watchObservedRunningTime="2026-07-09 04:07:32.097777076 +0000 UTC m=+451.372155027"
Jul 09 04:07:33.179481 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:33.179433    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e82a69-a681-44e2-ac54-8fb225295283" path="/var/lib/kubelet/pods/28e82a69-a681-44e2-ac54-8fb225295283/volumes"
Jul 09 04:07:36.004174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:36.004129    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-s5xbl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 09 04:07:36.004583 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:07:36.004229    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s5xbl" podUID="28e82a69-a681-44e2-ac54-8fb225295283" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.30:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 09 04:08:03.072006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:03.071972    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l882p"
Jul 09 04:08:03.074822 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:03.074799    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-th7dt"
Jul 09 04:08:04.335835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:04.335799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:04.388431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:04.388403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:16.830616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.830578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:16.891406 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.891382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:16.939637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.939612    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:16.940055 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.940041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e82a69-a681-44e2-ac54-8fb225295283" containerName="discovery"
Jul 09 04:08:16.940102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.940058    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e82a69-a681-44e2-ac54-8fb225295283" containerName="discovery"
Jul 09 04:08:16.940139 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.940130    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e82a69-a681-44e2-ac54-8fb225295283" containerName="discovery"
Jul 09 04:08:16.950858 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.950831    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:16.951006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.950927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:16.952947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.952929    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 04:08:16.953069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.952929    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xcrvl\""
Jul 09 04:08:16.953351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.953336    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 04:08:16.990230 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:16.990199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:17.079606 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.079580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-947jk\" (UniqueName: \"kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk\") pod \"kuadrant-operator-catalog-5ddbw\" (UID: \"83b45d4c-6463-4ac3-b4f1-25517be55b7c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:17.180428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.180406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-947jk\" (UniqueName: \"kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk\") pod \"kuadrant-operator-catalog-5ddbw\" (UID: \"83b45d4c-6463-4ac3-b4f1-25517be55b7c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:17.188085 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.188063    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-947jk\" (UniqueName: \"kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk\") pod \"kuadrant-operator-catalog-5ddbw\" (UID: \"83b45d4c-6463-4ac3-b4f1-25517be55b7c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:17.263251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.263225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:17.386988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.386945    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:17.410961 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.410941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:17.413267 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:17.413244    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b45d4c_6463_4ac3_b4f1_25517be55b7c.slice/crio-4702152a90c6f81655b8dfe77e0108fa9efec5d9fe7259b35da6fedf184c3d74 WatchSource:0}: Error finding container 4702152a90c6f81655b8dfe77e0108fa9efec5d9fe7259b35da6fedf184c3d74: Status 404 returned error can't find the container with id 4702152a90c6f81655b8dfe77e0108fa9efec5d9fe7259b35da6fedf184c3d74
Jul 09 04:08:17.521125 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.521059    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rkcwt"]
Jul 09 04:08:17.525809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.525794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:17.530672 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.530653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rkcwt"]
Jul 09 04:08:17.584750 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.584724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h849s\" (UniqueName: \"kubernetes.io/projected/9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8-kube-api-access-h849s\") pod \"kuadrant-operator-catalog-rkcwt\" (UID: \"9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:17.685624 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.685599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h849s\" (UniqueName: \"kubernetes.io/projected/9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8-kube-api-access-h849s\") pod \"kuadrant-operator-catalog-rkcwt\" (UID: \"9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:17.693098 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.693079    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h849s\" (UniqueName: \"kubernetes.io/projected/9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8-kube-api-access-h849s\") pod \"kuadrant-operator-catalog-rkcwt\" (UID: \"9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:17.836252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.836159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:17.961094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:17.961059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rkcwt"]
Jul 09 04:08:17.963241 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:17.963211    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8cc5f2_b8f8_4aa6_8282_266f6f4aecb8.slice/crio-39e426b80c9ac0bcb53c6cae2046936ea47d612ee50cbb6113559aa747af7901 WatchSource:0}: Error finding container 39e426b80c9ac0bcb53c6cae2046936ea47d612ee50cbb6113559aa747af7901: Status 404 returned error can't find the container with id 39e426b80c9ac0bcb53c6cae2046936ea47d612ee50cbb6113559aa747af7901
Jul 09 04:08:18.273891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:18.273857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt" event={"ID":"9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8","Type":"ContainerStarted","Data":"39e426b80c9ac0bcb53c6cae2046936ea47d612ee50cbb6113559aa747af7901"}
Jul 09 04:08:18.275181 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:18.275152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" event={"ID":"83b45d4c-6463-4ac3-b4f1-25517be55b7c","Type":"ContainerStarted","Data":"4702152a90c6f81655b8dfe77e0108fa9efec5d9fe7259b35da6fedf184c3d74"}
Jul 09 04:08:20.286688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.286649    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt" event={"ID":"9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8","Type":"ContainerStarted","Data":"e98dbb7d4bf02e432f61d49eb6493402afa8fa3388ffa2b17c3672857cffa0e7"}
Jul 09 04:08:20.288280 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.288255    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" podUID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" containerName="registry-server" containerID="cri-o://b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8" gracePeriod=2
Jul 09 04:08:20.288393 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.288260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" event={"ID":"83b45d4c-6463-4ac3-b4f1-25517be55b7c","Type":"ContainerStarted","Data":"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"}
Jul 09 04:08:20.307648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.307609    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt" podStartSLOduration=1.741788587 podStartE2EDuration="3.3075949s" totalImagesPullingTime="1.565806313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:20.304959375 +0000 UTC m=+499.579337306" watchObservedRunningTime="2026-07-09 04:08:20.3075949 +0000 UTC m=+499.581972808"
Jul 09 04:08:20.327268 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.327232    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" podStartSLOduration=2.211234575 podStartE2EDuration="4.327219741s" totalImagesPullingTime="2.115985166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:20.326684176 +0000 UTC m=+499.601062108" watchObservedRunningTime="2026-07-09 04:08:20.327219741 +0000 UTC m=+499.601597649"
Jul 09 04:08:20.532314 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.532294    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:20.612276 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.612253    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-947jk\" (UniqueName: \"kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk\") pod \"83b45d4c-6463-4ac3-b4f1-25517be55b7c\" (UID: \"83b45d4c-6463-4ac3-b4f1-25517be55b7c\") "
Jul 09 04:08:20.614306 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.614287    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk" (OuterVolumeSpecName: "kube-api-access-947jk") pod "83b45d4c-6463-4ac3-b4f1-25517be55b7c" (UID: "83b45d4c-6463-4ac3-b4f1-25517be55b7c"). InnerVolumeSpecName "kube-api-access-947jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:08:20.713833 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:20.713804    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-947jk\" (UniqueName: \"kubernetes.io/projected/83b45d4c-6463-4ac3-b4f1-25517be55b7c-kube-api-access-947jk\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:21.293571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.293544    2577 generic.go:356] "Generic (PLEG): container finished" podID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" containerID="b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8" exitCode=0
Jul 09 04:08:21.293953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.293602    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw"
Jul 09 04:08:21.293953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.293630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" event={"ID":"83b45d4c-6463-4ac3-b4f1-25517be55b7c","Type":"ContainerDied","Data":"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"}
Jul 09 04:08:21.293953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.293675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ddbw" event={"ID":"83b45d4c-6463-4ac3-b4f1-25517be55b7c","Type":"ContainerDied","Data":"4702152a90c6f81655b8dfe77e0108fa9efec5d9fe7259b35da6fedf184c3d74"}
Jul 09 04:08:21.293953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.293695    2577 scope.go:117] "RemoveContainer" containerID="b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"
Jul 09 04:08:21.302367 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.302353    2577 scope.go:117] "RemoveContainer" containerID="b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"
Jul 09 04:08:21.302626 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:08:21.302608    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8\": container with ID starting with b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8 not found: ID does not exist" containerID="b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"
Jul 09 04:08:21.302680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.302641    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8"} err="failed to get container status \"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8\": rpc error: code = NotFound desc = could not find container \"b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8\": container with ID starting with b3f991db2ee0fbd1c44230bc0058bb3beff040ad8c77cb7c054fa9ee7f9131a8 not found: ID does not exist"
Jul 09 04:08:21.308031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.308011    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:21.311308 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:21.311287    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ddbw"]
Jul 09 04:08:23.179231 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:23.179199    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" path="/var/lib/kubelet/pods/83b45d4c-6463-4ac3-b4f1-25517be55b7c/volumes"
Jul 09 04:08:23.505371 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:23.505294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:27.836989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:27.836953    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:27.837405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:27.837041    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:27.859638 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:27.859616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:28.351817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:28.351793    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rkcwt"
Jul 09 04:08:32.318863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.318833    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"]
Jul 09 04:08:32.319240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.319142    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" containerName="registry-server"
Jul 09 04:08:32.319240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.319152    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" containerName="registry-server"
Jul 09 04:08:32.319240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.319227    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b45d4c-6463-4ac3-b4f1-25517be55b7c" containerName="registry-server"
Jul 09 04:08:32.322569 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.322552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.324340 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.324316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lvckb\""
Jul 09 04:08:32.332710 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.332677    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"]
Jul 09 04:08:32.409892 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.409864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7j6\" (UniqueName: \"kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.410034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.409928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.410034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.409964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.510639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.510607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7j6\" (UniqueName: \"kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.510796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.510649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.510796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.510671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.510984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.510970    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.511063 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.511045    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.517826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.517804    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7j6\" (UniqueName: \"kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.633093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.633063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:32.757486 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.757445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"]
Jul 09 04:08:32.759530 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:32.759506    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05876de0_f871_48f9_85b2_eaf12b7b9541.slice/crio-b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d WatchSource:0}: Error finding container b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d: Status 404 returned error can't find the container with id b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d
Jul 09 04:08:32.919936 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.919877    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"]
Jul 09 04:08:32.923346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.923331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:32.930518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:32.930496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"]
Jul 09 04:08:33.015216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.015189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.015346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.015230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.015346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.015318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqnjm\" (UniqueName: \"kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.116538 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.116515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.116654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.116573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqnjm\" (UniqueName: \"kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.116654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.116604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.116914 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.116894    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.116951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.116921    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.124257 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.124234    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqnjm\" (UniqueName: \"kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.233708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.233641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:33.321173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.321141    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"]
Jul 09 04:08:33.326437 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.326417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.331962 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.331939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"]
Jul 09 04:08:33.353743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.353714    2577 generic.go:356] "Generic (PLEG): container finished" podID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerID="7b0fa9756dae4f763e7f22f972dc9c57433eee6948ed04bf7e2f1f0f54d0d930" exitCode=0
Jul 09 04:08:33.353891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.353759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx" event={"ID":"05876de0-f871-48f9-85b2-eaf12b7b9541","Type":"ContainerDied","Data":"7b0fa9756dae4f763e7f22f972dc9c57433eee6948ed04bf7e2f1f0f54d0d930"}
Jul 09 04:08:33.353891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.353800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx" event={"ID":"05876de0-f871-48f9-85b2-eaf12b7b9541","Type":"ContainerStarted","Data":"b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d"}
Jul 09 04:08:33.359245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.359195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"]
Jul 09 04:08:33.361364 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:33.361345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb812661_3716_4089_a69a_845bad3b0af1.slice/crio-85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff WatchSource:0}: Error finding container 85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff: Status 404 returned error can't find the container with id 85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff
Jul 09 04:08:33.418807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.418787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.418906 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.418849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzst\" (UniqueName: \"kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.418970 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.418930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.519413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.519353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.519413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.519405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzst\" (UniqueName: \"kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.519583 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.519455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.519763 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.519746    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.519842 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.519826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.527143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.527124    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzst\" (UniqueName: \"kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.637856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.637834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:33.725222 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.725195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"]
Jul 09 04:08:33.729883 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.729864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.737357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.737304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"]
Jul 09 04:08:33.766918 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.766893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"]
Jul 09 04:08:33.769347 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:33.769321    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4a3c01_b668_401d_a195_d486ffdc1b22.slice/crio-e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739 WatchSource:0}: Error finding container e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739: Status 404 returned error can't find the container with id e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739
Jul 09 04:08:33.822903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.822880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8cg4\" (UniqueName: \"kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.822989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.822939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.823062 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.823046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.923736 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.923712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.923835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.923764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.923835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.923800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8cg4\" (UniqueName: \"kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.924032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.924013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.924083 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.924067    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:33.931900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:33.931879    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8cg4\" (UniqueName: \"kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:34.041985 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.041913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:34.209501 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.206878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"]
Jul 09 04:08:34.210173 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:34.210148    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc389c884_7b60_4d13_82e4_ff110ce95d0f.slice/crio-30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e WatchSource:0}: Error finding container 30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e: Status 404 returned error can't find the container with id 30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e
Jul 09 04:08:34.360357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.360333    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerID="ba4e6f7b057189d4fdeb28af9f96544a0eb11c6c8be290a439378a018bb52ace" exitCode=0
Jul 09 04:08:34.360718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.360419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9" event={"ID":"ae4a3c01-b668-401d-a195-d486ffdc1b22","Type":"ContainerDied","Data":"ba4e6f7b057189d4fdeb28af9f96544a0eb11c6c8be290a439378a018bb52ace"}
Jul 09 04:08:34.360718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.360505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9" event={"ID":"ae4a3c01-b668-401d-a195-d486ffdc1b22","Type":"ContainerStarted","Data":"e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739"}
Jul 09 04:08:34.362335 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.362313    2577 generic.go:356] "Generic (PLEG): container finished" podID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerID="a1afd7944e24c98d8cec6e617ed44a1f6f4ac1350c68f7e5637f7fedc0b6b82c" exitCode=0
Jul 09 04:08:34.362410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.362394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx" event={"ID":"05876de0-f871-48f9-85b2-eaf12b7b9541","Type":"ContainerDied","Data":"a1afd7944e24c98d8cec6e617ed44a1f6f4ac1350c68f7e5637f7fedc0b6b82c"}
Jul 09 04:08:34.363960 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.363940    2577 generic.go:356] "Generic (PLEG): container finished" podID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerID="4736bed9a64ca51d67879ca9fbe1e2b59ea73fbbdf374193d4dea1aed2196555" exitCode=0
Jul 09 04:08:34.364059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.364014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n" event={"ID":"c389c884-7b60-4d13-82e4-ff110ce95d0f","Type":"ContainerDied","Data":"4736bed9a64ca51d67879ca9fbe1e2b59ea73fbbdf374193d4dea1aed2196555"}
Jul 09 04:08:34.364059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.364047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n" event={"ID":"c389c884-7b60-4d13-82e4-ff110ce95d0f","Type":"ContainerStarted","Data":"30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e"}
Jul 09 04:08:34.365760 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.365738    2577 generic.go:356] "Generic (PLEG): container finished" podID="db812661-3716-4089-a69a-845bad3b0af1" containerID="5d52fc8244e9384c5a0cc5470616f9102ae525cedf48b34bbab216241aa384e9" exitCode=0
Jul 09 04:08:34.365850 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.365814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z" event={"ID":"db812661-3716-4089-a69a-845bad3b0af1","Type":"ContainerDied","Data":"5d52fc8244e9384c5a0cc5470616f9102ae525cedf48b34bbab216241aa384e9"}
Jul 09 04:08:34.365940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:34.365847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z" event={"ID":"db812661-3716-4089-a69a-845bad3b0af1","Type":"ContainerStarted","Data":"85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff"}
Jul 09 04:08:35.373597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.373564    2577 generic.go:356] "Generic (PLEG): container finished" podID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerID="0bfb045851b9ea51ae5c2c43c84f3e9dc7f127f07637493e415a93dc1f054115" exitCode=0
Jul 09 04:08:35.373941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.373652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx" event={"ID":"05876de0-f871-48f9-85b2-eaf12b7b9541","Type":"ContainerDied","Data":"0bfb045851b9ea51ae5c2c43c84f3e9dc7f127f07637493e415a93dc1f054115"}
Jul 09 04:08:35.378817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.378793    2577 generic.go:356] "Generic (PLEG): container finished" podID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerID="5190eddcf1f093fe97a8fba009d7f6736d5be8867073d16351be3737cde01cb7" exitCode=0
Jul 09 04:08:35.378913 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.378875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n" event={"ID":"c389c884-7b60-4d13-82e4-ff110ce95d0f","Type":"ContainerDied","Data":"5190eddcf1f093fe97a8fba009d7f6736d5be8867073d16351be3737cde01cb7"}
Jul 09 04:08:35.380885 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.380864    2577 generic.go:356] "Generic (PLEG): container finished" podID="db812661-3716-4089-a69a-845bad3b0af1" containerID="0a9b90eb5bf24a10756da074c07f5cbb5d11abfbf45445f6054a80a17ca33941" exitCode=0
Jul 09 04:08:35.381008 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:35.380951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z" event={"ID":"db812661-3716-4089-a69a-845bad3b0af1","Type":"ContainerDied","Data":"0a9b90eb5bf24a10756da074c07f5cbb5d11abfbf45445f6054a80a17ca33941"}
Jul 09 04:08:36.387416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.387386    2577 generic.go:356] "Generic (PLEG): container finished" podID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerID="28c5c772bd401a3f3f2b36bc1e8bfedb5fabc0a07ac8d3e34a11bdf5189d467c" exitCode=0
Jul 09 04:08:36.387801 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.387456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n" event={"ID":"c389c884-7b60-4d13-82e4-ff110ce95d0f","Type":"ContainerDied","Data":"28c5c772bd401a3f3f2b36bc1e8bfedb5fabc0a07ac8d3e34a11bdf5189d467c"}
Jul 09 04:08:36.389391 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.389369    2577 generic.go:356] "Generic (PLEG): container finished" podID="db812661-3716-4089-a69a-845bad3b0af1" containerID="05be775aed1fba647706ca10f3934d61fbf343fbb5e6a6c690dacf58249ba9ec" exitCode=0
Jul 09 04:08:36.389539 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.389447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z" event={"ID":"db812661-3716-4089-a69a-845bad3b0af1","Type":"ContainerDied","Data":"05be775aed1fba647706ca10f3934d61fbf343fbb5e6a6c690dacf58249ba9ec"}
Jul 09 04:08:36.391193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.391173    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerID="aeca8f81b35d1b7b986a4c5374d39412d246b329b3ec2de9a84cb1ce7ea0396d" exitCode=0
Jul 09 04:08:36.391312 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.391245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9" event={"ID":"ae4a3c01-b668-401d-a195-d486ffdc1b22","Type":"ContainerDied","Data":"aeca8f81b35d1b7b986a4c5374d39412d246b329b3ec2de9a84cb1ce7ea0396d"}
Jul 09 04:08:36.525226 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.525193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:36.649865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.649794    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util\") pod \"05876de0-f871-48f9-85b2-eaf12b7b9541\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") "
Jul 09 04:08:36.650004 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.649891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x7j6\" (UniqueName: \"kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6\") pod \"05876de0-f871-48f9-85b2-eaf12b7b9541\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") "
Jul 09 04:08:36.650004 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.649914    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle\") pod \"05876de0-f871-48f9-85b2-eaf12b7b9541\" (UID: \"05876de0-f871-48f9-85b2-eaf12b7b9541\") "
Jul 09 04:08:36.650408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.650387    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle" (OuterVolumeSpecName: "bundle") pod "05876de0-f871-48f9-85b2-eaf12b7b9541" (UID: "05876de0-f871-48f9-85b2-eaf12b7b9541"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:36.651790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.651770    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6" (OuterVolumeSpecName: "kube-api-access-8x7j6") pod "05876de0-f871-48f9-85b2-eaf12b7b9541" (UID: "05876de0-f871-48f9-85b2-eaf12b7b9541"). InnerVolumeSpecName "kube-api-access-8x7j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:08:36.655001 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.654978    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util" (OuterVolumeSpecName: "util") pod "05876de0-f871-48f9-85b2-eaf12b7b9541" (UID: "05876de0-f871-48f9-85b2-eaf12b7b9541"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:36.751288 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.751265    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x7j6\" (UniqueName: \"kubernetes.io/projected/05876de0-f871-48f9-85b2-eaf12b7b9541-kube-api-access-8x7j6\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:36.751288 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.751287    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:36.751399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:36.751295    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05876de0-f871-48f9-85b2-eaf12b7b9541-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.397973 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.397939    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerID="9988743c4cdbc4d6a21025f877bc0c42796c70d7f358475f9785bbe35292c000" exitCode=0
Jul 09 04:08:37.398323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.398024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9" event={"ID":"ae4a3c01-b668-401d-a195-d486ffdc1b22","Type":"ContainerDied","Data":"9988743c4cdbc4d6a21025f877bc0c42796c70d7f358475f9785bbe35292c000"}
Jul 09 04:08:37.399877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.399853    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx"
Jul 09 04:08:37.399997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.399883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx" event={"ID":"05876de0-f871-48f9-85b2-eaf12b7b9541","Type":"ContainerDied","Data":"b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d"}
Jul 09 04:08:37.399997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.399913    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9c2bef19a3b7ad4cc813a9dfd34f92c084bb5b460563d073490e20b1ffc025d"
Jul 09 04:08:37.526283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.526263    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:37.557743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.557720    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:37.658397 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqnjm\" (UniqueName: \"kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm\") pod \"db812661-3716-4089-a69a-845bad3b0af1\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") "
Jul 09 04:08:37.658564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8cg4\" (UniqueName: \"kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4\") pod \"c389c884-7b60-4d13-82e4-ff110ce95d0f\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") "
Jul 09 04:08:37.658564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle\") pod \"db812661-3716-4089-a69a-845bad3b0af1\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") "
Jul 09 04:08:37.658564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle\") pod \"c389c884-7b60-4d13-82e4-ff110ce95d0f\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") "
Jul 09 04:08:37.658564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658540    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util\") pod \"db812661-3716-4089-a69a-845bad3b0af1\" (UID: \"db812661-3716-4089-a69a-845bad3b0af1\") "
Jul 09 04:08:37.658564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.658564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util\") pod \"c389c884-7b60-4d13-82e4-ff110ce95d0f\" (UID: \"c389c884-7b60-4d13-82e4-ff110ce95d0f\") "
Jul 09 04:08:37.659097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.659060    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle" (OuterVolumeSpecName: "bundle") pod "db812661-3716-4089-a69a-845bad3b0af1" (UID: "db812661-3716-4089-a69a-845bad3b0af1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:37.659260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.659237    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle" (OuterVolumeSpecName: "bundle") pod "c389c884-7b60-4d13-82e4-ff110ce95d0f" (UID: "c389c884-7b60-4d13-82e4-ff110ce95d0f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:37.660495 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.660454    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4" (OuterVolumeSpecName: "kube-api-access-l8cg4") pod "c389c884-7b60-4d13-82e4-ff110ce95d0f" (UID: "c389c884-7b60-4d13-82e4-ff110ce95d0f"). InnerVolumeSpecName "kube-api-access-l8cg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:08:37.660813 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.660794    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm" (OuterVolumeSpecName: "kube-api-access-wqnjm") pod "db812661-3716-4089-a69a-845bad3b0af1" (UID: "db812661-3716-4089-a69a-845bad3b0af1"). InnerVolumeSpecName "kube-api-access-wqnjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:08:37.664352 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.664318    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util" (OuterVolumeSpecName: "util") pod "db812661-3716-4089-a69a-845bad3b0af1" (UID: "db812661-3716-4089-a69a-845bad3b0af1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:37.664736 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.664715    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util" (OuterVolumeSpecName: "util") pod "c389c884-7b60-4d13-82e4-ff110ce95d0f" (UID: "c389c884-7b60-4d13-82e4-ff110ce95d0f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759757    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8cg4\" (UniqueName: \"kubernetes.io/projected/c389c884-7b60-4d13-82e4-ff110ce95d0f-kube-api-access-l8cg4\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759793    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759804    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759811    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db812661-3716-4089-a69a-845bad3b0af1-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759819    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c389c884-7b60-4d13-82e4-ff110ce95d0f-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:37.759826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:37.759826    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqnjm\" (UniqueName: \"kubernetes.io/projected/db812661-3716-4089-a69a-845bad3b0af1-kube-api-access-wqnjm\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:38.406283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.406257    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n"
Jul 09 04:08:38.406696 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.406255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n" event={"ID":"c389c884-7b60-4d13-82e4-ff110ce95d0f","Type":"ContainerDied","Data":"30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e"}
Jul 09 04:08:38.406696 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.406382    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30cd96e8e7e2f35358bdd5f1c5cdac45c06fe1cfe9f9efe4afd1aa2ce512065e"
Jul 09 04:08:38.408164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.408143    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z"
Jul 09 04:08:38.408164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.408150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z" event={"ID":"db812661-3716-4089-a69a-845bad3b0af1","Type":"ContainerDied","Data":"85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff"}
Jul 09 04:08:38.408313 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.408187    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85bda96ab6e430c8e4f2864f95b2465b8eeece159714532624be0ed7da781aff"
Jul 09 04:08:38.532065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.532041    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:38.664810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.664737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util\") pod \"ae4a3c01-b668-401d-a195-d486ffdc1b22\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") "
Jul 09 04:08:38.664934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.664812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mzst\" (UniqueName: \"kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst\") pod \"ae4a3c01-b668-401d-a195-d486ffdc1b22\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") "
Jul 09 04:08:38.664934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.664861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle\") pod \"ae4a3c01-b668-401d-a195-d486ffdc1b22\" (UID: \"ae4a3c01-b668-401d-a195-d486ffdc1b22\") "
Jul 09 04:08:38.665313 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.665282    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle" (OuterVolumeSpecName: "bundle") pod "ae4a3c01-b668-401d-a195-d486ffdc1b22" (UID: "ae4a3c01-b668-401d-a195-d486ffdc1b22"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:38.666766 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.666739    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst" (OuterVolumeSpecName: "kube-api-access-2mzst") pod "ae4a3c01-b668-401d-a195-d486ffdc1b22" (UID: "ae4a3c01-b668-401d-a195-d486ffdc1b22"). InnerVolumeSpecName "kube-api-access-2mzst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:08:38.672250 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.672227    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util" (OuterVolumeSpecName: "util") pod "ae4a3c01-b668-401d-a195-d486ffdc1b22" (UID: "ae4a3c01-b668-401d-a195-d486ffdc1b22"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:08:38.766291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.766267    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mzst\" (UniqueName: \"kubernetes.io/projected/ae4a3c01-b668-401d-a195-d486ffdc1b22-kube-api-access-2mzst\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:38.766291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.766288    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-bundle\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:38.766419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:38.766297    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae4a3c01-b668-401d-a195-d486ffdc1b22-util\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:08:39.414340 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:39.414309    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9"
Jul 09 04:08:39.414739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:39.414311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9" event={"ID":"ae4a3c01-b668-401d-a195-d486ffdc1b22","Type":"ContainerDied","Data":"e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739"}
Jul 09 04:08:39.414739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:39.414431    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e172fb263e8097048e3e4695fb4925c1bab9a89990ba29c616e0730151a44739"
Jul 09 04:08:42.198175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:42.198135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:42.250746 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:42.250711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:45.962266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:45.962234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:48.398143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398114    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"]
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398484    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="util"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398496    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="util"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398504    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398510    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398517    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398523    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398534    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398539    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398553    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398560    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398569    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="util"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398574    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="util"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398580    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398585    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398592    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398596    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="extract"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398602    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398607    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="pull"
Jul 09 04:08:48.398605 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398616    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="util"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398621    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="util"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398627    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="pull"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398651    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="pull"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398661    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="util"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398675    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="util"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398744    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="db812661-3716-4089-a69a-845bad3b0af1" containerName="extract"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398753    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4a3c01-b668-401d-a195-d486ffdc1b22" containerName="extract"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398760    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="05876de0-f871-48f9-85b2-eaf12b7b9541" containerName="extract"
Jul 09 04:08:48.399173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.398765    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c389c884-7b60-4d13-82e4-ff110ce95d0f" containerName="extract"
Jul 09 04:08:48.402161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.402146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:48.404034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.404018    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 04:08:48.404121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.404080    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-b8s76\""
Jul 09 04:08:48.412251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.412231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"]
Jul 09 04:08:48.450073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.450050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcq9l\" (UniqueName: \"kubernetes.io/projected/3ef83895-c441-4770-ae66-3176275a0a98-kube-api-access-lcq9l\") pod \"dns-operator-controller-manager-864dd78c4c-sd6h9\" (UID: \"3ef83895-c441-4770-ae66-3176275a0a98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:48.550798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.550772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcq9l\" (UniqueName: \"kubernetes.io/projected/3ef83895-c441-4770-ae66-3176275a0a98-kube-api-access-lcq9l\") pod \"dns-operator-controller-manager-864dd78c4c-sd6h9\" (UID: \"3ef83895-c441-4770-ae66-3176275a0a98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:48.557699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.557681    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcq9l\" (UniqueName: \"kubernetes.io/projected/3ef83895-c441-4770-ae66-3176275a0a98-kube-api-access-lcq9l\") pod \"dns-operator-controller-manager-864dd78c4c-sd6h9\" (UID: \"3ef83895-c441-4770-ae66-3176275a0a98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:48.712203 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.712125    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:48.843870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:48.843847    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"]
Jul 09 04:08:48.845842 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:48.845812    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef83895_c441_4770_ae66_3176275a0a98.slice/crio-9cdd997a6179ab17f64c24b0186908e862501c83736e7a04fa07440dc1c9aa76 WatchSource:0}: Error finding container 9cdd997a6179ab17f64c24b0186908e862501c83736e7a04fa07440dc1c9aa76: Status 404 returned error can't find the container with id 9cdd997a6179ab17f64c24b0186908e862501c83736e7a04fa07440dc1c9aa76
Jul 09 04:08:49.468662 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:49.468629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9" event={"ID":"3ef83895-c441-4770-ae66-3176275a0a98","Type":"ContainerStarted","Data":"9cdd997a6179ab17f64c24b0186908e862501c83736e7a04fa07440dc1c9aa76"}
Jul 09 04:08:49.958026 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:49.957988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:51.482175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.482138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9" event={"ID":"3ef83895-c441-4770-ae66-3176275a0a98","Type":"ContainerStarted","Data":"61cf8ebd55d7dfe87721952174016c61da23ddaeb0a1b5984f438a8556f30ef0"}
Jul 09 04:08:51.482571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.482209    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:08:51.499005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.498960    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9" podStartSLOduration=1.43473865 podStartE2EDuration="3.498945448s" totalImagesPullingTime="2.064206798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:51.496444912 +0000 UTC m=+530.770822845" watchObservedRunningTime="2026-07-09 04:08:51.498945448 +0000 UTC m=+530.773323356"
Jul 09 04:08:51.867951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.867916    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"]
Jul 09 04:08:51.871522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.871493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:51.873561 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.873543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t2p4g\""
Jul 09 04:08:51.887887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.887868    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"]
Jul 09 04:08:51.980350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:51.980321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnwhr\" (UniqueName: \"kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr\") pod \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" (UID: \"10d00343-4c73-4709-ab2c-9548b49551e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:52.081206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.081165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnwhr\" (UniqueName: \"kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr\") pod \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" (UID: \"10d00343-4c73-4709-ab2c-9548b49551e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:52.092180 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.092158    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnwhr\" (UniqueName: \"kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr\") pod \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" (UID: \"10d00343-4c73-4709-ab2c-9548b49551e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:52.182357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.182291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:52.317671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.317650    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"]
Jul 09 04:08:52.319318 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:52.319288    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d00343_4c73_4709_ab2c_9548b49551e1.slice/crio-25f6007f82a967539913882d4f7730a114e7d9f5f3a463aae84508edb8b8e605 WatchSource:0}: Error finding container 25f6007f82a967539913882d4f7730a114e7d9f5f3a463aae84508edb8b8e605: Status 404 returned error can't find the container with id 25f6007f82a967539913882d4f7730a114e7d9f5f3a463aae84508edb8b8e605
Jul 09 04:08:52.489047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.488962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw" event={"ID":"10d00343-4c73-4709-ab2c-9548b49551e1","Type":"ContainerStarted","Data":"25f6007f82a967539913882d4f7730a114e7d9f5f3a463aae84508edb8b8e605"}
Jul 09 04:08:52.590219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.590180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:52.813699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:52.813539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:53.661905 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:53.661825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:53.778799 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:53.778756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:53.864237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:53.864194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:54.502206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:54.502172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw" event={"ID":"10d00343-4c73-4709-ab2c-9548b49551e1","Type":"ContainerStarted","Data":"24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f"}
Jul 09 04:08:54.502377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:54.502278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:08:54.520773 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:54.520722    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw" podStartSLOduration=1.5643518090000001 podStartE2EDuration="3.520709511s" totalImagesPullingTime="1.956357702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:54.520209807 +0000 UTC m=+533.794587734" watchObservedRunningTime="2026-07-09 04:08:54.520709511 +0000 UTC m=+533.795087419"
Jul 09 04:08:55.591714 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:55.591681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:56.049656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:56.049619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:57.056022 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.055987    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"]
Jul 09 04:08:57.059720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.059703    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.062159 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.062138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kmxl5\""
Jul 09 04:08:57.074138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.074113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"]
Jul 09 04:08:57.120328 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.120300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsnff\" (UniqueName: \"kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.120519 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.120342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.153522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.153490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:57.221032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.220991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnff\" (UniqueName: \"kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.221032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.221035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.221405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.221388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.230102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.230078    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnff\" (UniqueName: \"kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff\") pod \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.369979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.369945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:08:57.502850 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.502823    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"]
Jul 09 04:08:57.504832 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:08:57.504795    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b50a1e7_bb4c_41c6_8cfb_3900f808e9bb.slice/crio-2e0b6f0d23059cc1bff91d99f7db95865f7581e3bf6f2e89cd8073ae5b43b15f WatchSource:0}: Error finding container 2e0b6f0d23059cc1bff91d99f7db95865f7581e3bf6f2e89cd8073ae5b43b15f: Status 404 returned error can't find the container with id 2e0b6f0d23059cc1bff91d99f7db95865f7581e3bf6f2e89cd8073ae5b43b15f
Jul 09 04:08:57.518873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.518836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc" event={"ID":"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb","Type":"ContainerStarted","Data":"2e0b6f0d23059cc1bff91d99f7db95865f7581e3bf6f2e89cd8073ae5b43b15f"}
Jul 09 04:08:57.552144 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.552113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:57.790503 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:57.790378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:59.660908 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.660871    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"]
Jul 09 04:08:59.664720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.664699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:08:59.668794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.668768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c2fdr\""
Jul 09 04:08:59.683350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.683326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"]
Jul 09 04:08:59.744363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.744328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/1b8c1d34-b174-4002-897d-2b14cbae6c0d-kube-api-access-5tgql\") pod \"authorino-operator-75cc8d5cb7-hmk86\" (UID: \"1b8c1d34-b174-4002-897d-2b14cbae6c0d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:08:59.845183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.845148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/1b8c1d34-b174-4002-897d-2b14cbae6c0d-kube-api-access-5tgql\") pod \"authorino-operator-75cc8d5cb7-hmk86\" (UID: \"1b8c1d34-b174-4002-897d-2b14cbae6c0d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:08:59.856411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.856371    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/1b8c1d34-b174-4002-897d-2b14cbae6c0d-kube-api-access-5tgql\") pod \"authorino-operator-75cc8d5cb7-hmk86\" (UID: \"1b8c1d34-b174-4002-897d-2b14cbae6c0d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:08:59.977957 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:08:59.977882    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:09:00.130722 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:00.130698    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"]
Jul 09 04:09:00.133122 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:00.133098    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8c1d34_b174_4002_897d_2b14cbae6c0d.slice/crio-8c0a4488d85968c03e2a481f9a9d5eb9adf14ec94ccb298e61c0e603f7291f19 WatchSource:0}: Error finding container 8c0a4488d85968c03e2a481f9a9d5eb9adf14ec94ccb298e61c0e603f7291f19: Status 404 returned error can't find the container with id 8c0a4488d85968c03e2a481f9a9d5eb9adf14ec94ccb298e61c0e603f7291f19
Jul 09 04:09:00.536909 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:00.536874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86" event={"ID":"1b8c1d34-b174-4002-897d-2b14cbae6c0d","Type":"ContainerStarted","Data":"8c0a4488d85968c03e2a481f9a9d5eb9adf14ec94ccb298e61c0e603f7291f19"}
Jul 09 04:09:01.112891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:01.112864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:01.210077 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:01.210051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:02.491682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:02.491653    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sd6h9"
Jul 09 04:09:03.562031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:03.561995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc" event={"ID":"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb","Type":"ContainerStarted","Data":"3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19"}
Jul 09 04:09:03.562517 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:03.562143    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:03.579245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:03.579195    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc" podStartSLOduration=1.07741455 podStartE2EDuration="6.579176497s" totalImagesPullingTime="5.501761947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:03.577047139 +0000 UTC m=+542.851425082" watchObservedRunningTime="2026-07-09 04:09:03.579176497 +0000 UTC m=+542.853554406"
Jul 09 04:09:04.156566 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:04.156520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:04.256860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:04.256795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:05.509438 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:05.509410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:05.574047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:05.574017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86" event={"ID":"1b8c1d34-b174-4002-897d-2b14cbae6c0d","Type":"ContainerStarted","Data":"2b0c73c56e0610e340687149414b5c97078aa991daeca071af19ee75904d485f"}
Jul 09 04:09:05.574205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:05.574155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:09:05.589138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:05.589096    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86" podStartSLOduration=1.7909153 podStartE2EDuration="6.589080419s" totalImagesPullingTime="4.798165119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:05.587767587 +0000 UTC m=+544.862145516" watchObservedRunningTime="2026-07-09 04:09:05.589080419 +0000 UTC m=+544.863458327"
Jul 09 04:09:06.085861 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:06.085827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:06.283619 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:06.283583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:07.754364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:07.754330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:07.855897 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:07.855863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:14.570062 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:14.570035    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:15.561856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.561785    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"]
Jul 09 04:09:15.566874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.566854    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.595419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.595399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"]
Jul 09 04:09:15.687072 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.687041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtc7f\" (UniqueName: \"kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.687246 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.687088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.787606 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.787570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.787770 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.787669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc7f\" (UniqueName: \"kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.787988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.787961    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.805820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.805791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtc7f\" (UniqueName: \"kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:15.876173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:15.876146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:16.016920 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.016896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"]
Jul 09 04:09:16.018665 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:16.018634    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14644cd7_91b1_4860_acbf_0b1d2fefba31.slice/crio-ff77bcdbe9a835a390504061c89036ed6ffb7a4ed1982ef0f918bd8da9898fc6 WatchSource:0}: Error finding container ff77bcdbe9a835a390504061c89036ed6ffb7a4ed1982ef0f918bd8da9898fc6: Status 404 returned error can't find the container with id ff77bcdbe9a835a390504061c89036ed6ffb7a4ed1982ef0f918bd8da9898fc6
Jul 09 04:09:16.085221 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.085199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:16.187522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.187409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:16.286671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.286632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:16.296198 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.296159    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"]
Jul 09 04:09:16.300287 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.300264    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"]
Jul 09 04:09:16.309889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.309865    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"]
Jul 09 04:09:16.310109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.310065    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" containerName="manager" containerID="cri-o://3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19" gracePeriod=2
Jul 09 04:09:16.316800 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.316746    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"]
Jul 09 04:09:16.318442 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.318345    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-279xc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.338936 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.338900    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"]
Jul 09 04:09:16.339538 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.339181    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" containerName="manager" containerID="cri-o://24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f" gracePeriod=2
Jul 09 04:09:16.340901 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.340864    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.341853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.341808    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:16.342436 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342403    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" containerName="manager"
Jul 09 04:09:16.342436 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342429    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" containerName="manager"
Jul 09 04:09:16.342436 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.342419    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.342767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342463    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" containerName="manager"
Jul 09 04:09:16.342767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342501    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" containerName="manager"
Jul 09 04:09:16.342767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342589    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" containerName="manager"
Jul 09 04:09:16.342767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.342601    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" containerName="manager"
Jul 09 04:09:16.346077 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.346055    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"]
Jul 09 04:09:16.346198 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.346174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.354550 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.354527    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:16.355104 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.355082    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" containerName="manager"
Jul 09 04:09:16.355193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.355113    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" containerName="manager"
Jul 09 04:09:16.355248 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.355235    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" containerName="manager"
Jul 09 04:09:16.358565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.358546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.362311 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.362288    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:16.379860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.379831    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"]
Jul 09 04:09:16.383261 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.383244    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:16.390994 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.390969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:16.392858 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.392837    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:16.401600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.401579    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"]
Jul 09 04:09:16.411137 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.411109    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.412645 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.412608    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.414623 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.414593    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.457902 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.457871    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.495589 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.495551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqzk\" (UniqueName: \"kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.495589 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.495588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.495836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.495633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpz5\" (UniqueName: \"kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.495836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.495676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spjc8\" (UniqueName: \"kubernetes.io/projected/d5a13346-95df-4cd5-8821-077331e64d81-kube-api-access-spjc8\") pod \"limitador-operator-controller-manager-859d7b78bd-qjsfq\" (UID: \"d5a13346-95df-4cd5-8821-077331e64d81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:16.495836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.495748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.569336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.569311    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.571526 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.571450    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.573080 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.573058    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.577431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.577413    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.579073 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.579047    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.580440 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.580418    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.581911 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.581893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hmk86"
Jul 09 04:09:16.596763 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.596731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.596789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqzk\" (UniqueName: \"kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.596818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.596870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czpz5\" (UniqueName: \"kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.596934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spjc8\" (UniqueName: \"kubernetes.io/projected/d5a13346-95df-4cd5-8821-077331e64d81-kube-api-access-spjc8\") pod \"limitador-operator-controller-manager-859d7b78bd-qjsfq\" (UID: \"d5a13346-95df-4cd5-8821-077331e64d81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.597109    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.597218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.597188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.601070 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.601045    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.602452 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.602430    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.609870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.609841    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpz5\" (UniqueName: \"kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hqhw4\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.610019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.609917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spjc8\" (UniqueName: \"kubernetes.io/projected/d5a13346-95df-4cd5-8821-077331e64d81-kube-api-access-spjc8\") pod \"limitador-operator-controller-manager-859d7b78bd-qjsfq\" (UID: \"d5a13346-95df-4cd5-8821-077331e64d81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:16.612709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.612604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqzk\" (UniqueName: \"kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hshc7\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.635409 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.635375    2577 generic.go:356] "Generic (PLEG): container finished" podID="10d00343-4c73-4709-ab2c-9548b49551e1" containerID="24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f" exitCode=0
Jul 09 04:09:16.635631 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.635441    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.635631 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.635488    2577 scope.go:117] "RemoveContainer" containerID="24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f"
Jul 09 04:09:16.637304 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.637282    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" containerID="3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19" exitCode=0
Jul 09 04:09:16.637422 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.637355    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.637692 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.637667    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.639037 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.639006    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.639782 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.639762    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m" container="manager"
Jul 09 04:09:16.640448 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.640418    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.641734 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.641710    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.642858 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.642835    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.645101 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.645072    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.645906 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.645882    2577 scope.go:117] "RemoveContainer" containerID="24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f"
Jul 09 04:09:16.646187 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.646166    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f\": container with ID starting with 24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f not found: ID does not exist" containerID="24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f"
Jul 09 04:09:16.646274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.646203    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f"} err="failed to get container status \"24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f\": rpc error: code = NotFound desc = could not find container \"24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f\": container with ID starting with 24b82da9e1c7dd74e5a59b9e1e79ebfa071c0be16d935de36de84e2e287d1f2f not found: ID does not exist"
Jul 09 04:09:16.646274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.646224    2577 scope.go:117] "RemoveContainer" containerID="3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19"
Jul 09 04:09:16.646458 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.646432    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:16.656858 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.656834    2577 scope.go:117] "RemoveContainer" containerID="3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19"
Jul 09 04:09:16.657139 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.657121    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19\": container with ID starting with 3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19 not found: ID does not exist" containerID="3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19"
Jul 09 04:09:16.657197 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.657147    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19"} err="failed to get container status \"3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19\": rpc error: code = NotFound desc = could not find container \"3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19\": container with ID starting with 3cd29b4acb3d5955b74f5102827e059b9c413a5a968e7054236fd31a291cce19 not found: ID does not exist"
Jul 09 04:09:16.698229 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.698148    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume\") pod \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") "
Jul 09 04:09:16.698229 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.698212    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnwhr\" (UniqueName: \"kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr\") pod \"10d00343-4c73-4709-ab2c-9548b49551e1\" (UID: \"10d00343-4c73-4709-ab2c-9548b49551e1\") "
Jul 09 04:09:16.698426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.698282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsnff\" (UniqueName: \"kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff\") pod \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\" (UID: \"0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb\") "
Jul 09 04:09:16.698815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.698792    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" (UID: "0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:09:16.700272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.700253    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr" (OuterVolumeSpecName: "kube-api-access-wnwhr") pod "10d00343-4c73-4709-ab2c-9548b49551e1" (UID: "10d00343-4c73-4709-ab2c-9548b49551e1"). InnerVolumeSpecName "kube-api-access-wnwhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:16.700338 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.700273    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff" (OuterVolumeSpecName: "kube-api-access-wsnff") pod "0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" (UID: "0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb"). InnerVolumeSpecName "kube-api-access-wsnff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:16.720445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.720425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:16.731515 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.731498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:16.737421 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.737401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:16.799999 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.799962    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsnff\" (UniqueName: \"kubernetes.io/projected/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-kube-api-access-wsnff\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:16.799999 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.800000    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb-extensions-socket-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:16.800232 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.800015    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnwhr\" (UniqueName: \"kubernetes.io/projected/10d00343-4c73-4709-ab2c-9548b49551e1-kube-api-access-wnwhr\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:16.897502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:16.897454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:16.900072 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:16.900043    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd845f4bd_f72d_49d0_9817_b42d6e674486.slice/crio-360b78cfdfa866dbceb0154ce0e85ed6a4ac62994fcc7f0177381c98569ce3e9 WatchSource:0}: Error finding container 360b78cfdfa866dbceb0154ce0e85ed6a4ac62994fcc7f0177381c98569ce3e9: Status 404 returned error can't find the container with id 360b78cfdfa866dbceb0154ce0e85ed6a4ac62994fcc7f0177381c98569ce3e9
Jul 09 04:09:16.949937 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.949868    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:16.951877 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.951853    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:16.953306 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.953287    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.954670 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.954643    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:16.956135 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.956115    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l9tcw\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l9tcw"
Jul 09 04:09:16.957438 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:16.957413    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-279xc\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-279xc"
Jul 09 04:09:17.123744 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.123719    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"]
Jul 09 04:09:17.125444 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:17.125423    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a13346_95df_4cd5_8821_077331e64d81.slice/crio-f10a7d30d2d359e690a7fea0d86f2eed814221bcaafc84a56536cdd3d03542be WatchSource:0}: Error finding container f10a7d30d2d359e690a7fea0d86f2eed814221bcaafc84a56536cdd3d03542be: Status 404 returned error can't find the container with id f10a7d30d2d359e690a7fea0d86f2eed814221bcaafc84a56536cdd3d03542be
Jul 09 04:09:17.136688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.136666    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:17.140229 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:17.140206    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7902334_164a_4b4c_9c6d_d0fce5a2271b.slice/crio-895982cb0d47a5e822329f41bb22971f4d5dc3f2cad07f291550ccd72e13a6de WatchSource:0}: Error finding container 895982cb0d47a5e822329f41bb22971f4d5dc3f2cad07f291550ccd72e13a6de: Status 404 returned error can't find the container with id 895982cb0d47a5e822329f41bb22971f4d5dc3f2cad07f291550ccd72e13a6de
Jul 09 04:09:17.181445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.181422    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb" path="/var/lib/kubelet/pods/0b50a1e7-bb4c-41c6-8cfb-3900f808e9bb/volumes"
Jul 09 04:09:17.181786 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.181772    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d00343-4c73-4709-ab2c-9548b49551e1" path="/var/lib/kubelet/pods/10d00343-4c73-4709-ab2c-9548b49551e1/volumes"
Jul 09 04:09:17.647643 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.647607    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" containerName="manager" containerID="cri-o://bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f" gracePeriod=2
Jul 09 04:09:17.649126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.649100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq" event={"ID":"d5a13346-95df-4cd5-8821-077331e64d81","Type":"ContainerStarted","Data":"6f6be6dc2d6364d3765518e13eefcef6793a5de698c86895612bc08c81896f32"}
Jul 09 04:09:17.649303 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.649281    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:17.649360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.649323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq" event={"ID":"d5a13346-95df-4cd5-8821-077331e64d81","Type":"ContainerStarted","Data":"f10a7d30d2d359e690a7fea0d86f2eed814221bcaafc84a56536cdd3d03542be"}
Jul 09 04:09:17.650947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.650923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" event={"ID":"d845f4bd-f72d-49d0-9817-b42d6e674486","Type":"ContainerStarted","Data":"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"}
Jul 09 04:09:17.651073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.650961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" event={"ID":"d845f4bd-f72d-49d0-9817-b42d6e674486","Type":"ContainerStarted","Data":"360b78cfdfa866dbceb0154ce0e85ed6a4ac62994fcc7f0177381c98569ce3e9"}
Jul 09 04:09:17.651073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.651010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:17.652815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.652737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" event={"ID":"b7902334-164a-4b4c-9c6d-d0fce5a2271b","Type":"ContainerStarted","Data":"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"}
Jul 09 04:09:17.652815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.652774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" event={"ID":"b7902334-164a-4b4c-9c6d-d0fce5a2271b","Type":"ContainerStarted","Data":"895982cb0d47a5e822329f41bb22971f4d5dc3f2cad07f291550ccd72e13a6de"}
Jul 09 04:09:17.652918 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.652825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:17.669350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.669308    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq" podStartSLOduration=1.669294396 podStartE2EDuration="1.669294396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:17.666781142 +0000 UTC m=+556.941159050" watchObservedRunningTime="2026-07-09 04:09:17.669294396 +0000 UTC m=+556.943672303"
Jul 09 04:09:17.702901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.702855    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" podStartSLOduration=1.702840252 podStartE2EDuration="1.702840252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:17.702254579 +0000 UTC m=+556.976632516" watchObservedRunningTime="2026-07-09 04:09:17.702840252 +0000 UTC m=+556.977218159"
Jul 09 04:09:17.703392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.703372    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" podStartSLOduration=1.703367025 podStartE2EDuration="1.703367025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:17.684307839 +0000 UTC m=+556.958685752" watchObservedRunningTime="2026-07-09 04:09:17.703367025 +0000 UTC m=+556.977744933"
Jul 09 04:09:17.894003 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:17.893975    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:17.895929 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:17.895905    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:18.008909 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.008823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume\") pod \"14644cd7-91b1-4860-acbf-0b1d2fefba31\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") "
Jul 09 04:09:18.008909 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.008898    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtc7f\" (UniqueName: \"kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f\") pod \"14644cd7-91b1-4860-acbf-0b1d2fefba31\" (UID: \"14644cd7-91b1-4860-acbf-0b1d2fefba31\") "
Jul 09 04:09:18.009133 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.009040    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "14644cd7-91b1-4860-acbf-0b1d2fefba31" (UID: "14644cd7-91b1-4860-acbf-0b1d2fefba31"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:09:18.010921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.010901    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f" (OuterVolumeSpecName: "kube-api-access-wtc7f") pod "14644cd7-91b1-4860-acbf-0b1d2fefba31" (UID: "14644cd7-91b1-4860-acbf-0b1d2fefba31"). InnerVolumeSpecName "kube-api-access-wtc7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:18.110284 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.110255    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtc7f\" (UniqueName: \"kubernetes.io/projected/14644cd7-91b1-4860-acbf-0b1d2fefba31-kube-api-access-wtc7f\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:18.110435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.110281    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14644cd7-91b1-4860-acbf-0b1d2fefba31-extensions-socket-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:18.661323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.661291    2577 generic.go:356] "Generic (PLEG): container finished" podID="14644cd7-91b1-4860-acbf-0b1d2fefba31" containerID="bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f" exitCode=2
Jul 09 04:09:18.661816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.661348    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:18.661816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.661396    2577 scope.go:117] "RemoveContainer" containerID="bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f"
Jul 09 04:09:18.663234 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:18.663209    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:18.671692 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:18.671666    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sck9m\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sck9m"
Jul 09 04:09:18.673491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.673448    2577 scope.go:117] "RemoveContainer" containerID="bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f"
Jul 09 04:09:18.673735 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:18.673719    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f\": container with ID starting with bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f not found: ID does not exist" containerID="bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f"
Jul 09 04:09:18.673789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.673742    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f"} err="failed to get container status \"bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f\": rpc error: code = NotFound desc = could not find container \"bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f\": container with ID starting with bd52f238634cc705f9cfbfa5926b0f6b69663419a0c675cd7a133488c9cd405f not found: ID does not exist"
Jul 09 04:09:18.830977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.830939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:18.939829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:18.939735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:19.179721 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:19.179686    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14644cd7-91b1-4860-acbf-0b1d2fefba31" path="/var/lib/kubelet/pods/14644cd7-91b1-4860-acbf-0b1d2fefba31/volumes"
Jul 09 04:09:22.007656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:22.007619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:22.107431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:22.107396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:25.608420 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:25.608385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:25.663758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:25.663731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:28.664086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.664056    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:28.664503 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.664129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:28.664503 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.664255    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qjsfq"
Jul 09 04:09:28.738364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.738332    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:28.738550 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.738532    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" podUID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" containerName="manager" containerID="cri-o://63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7" gracePeriod=10
Jul 09 04:09:28.942676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.942586    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:09:28.947172 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.947149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:28.961910 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:28.961885    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:09:29.003358 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.003336    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:29.109380 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume\") pod \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") "
Jul 09 04:09:29.109613 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109535    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hqzk\" (UniqueName: \"kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk\") pod \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\" (UID: \"b7902334-164a-4b4c-9c6d-d0fce5a2271b\") "
Jul 09 04:09:29.109735 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhdgt\" (UniqueName: \"kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.109796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109738    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b7902334-164a-4b4c-9c6d-d0fce5a2271b" (UID: "b7902334-164a-4b4c-9c6d-d0fce5a2271b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:09:29.109796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.109904 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.109865    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7902334-164a-4b4c-9c6d-d0fce5a2271b-extensions-socket-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:29.111544 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.111522    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk" (OuterVolumeSpecName: "kube-api-access-7hqzk") pod "b7902334-164a-4b4c-9c6d-d0fce5a2271b" (UID: "b7902334-164a-4b4c-9c6d-d0fce5a2271b"). InnerVolumeSpecName "kube-api-access-7hqzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:29.211078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.211009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.211188 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.211096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhdgt\" (UniqueName: \"kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.211188 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.211125    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hqzk\" (UniqueName: \"kubernetes.io/projected/b7902334-164a-4b4c-9c6d-d0fce5a2271b-kube-api-access-7hqzk\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:29.211392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.211374    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.219570 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.219548    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhdgt\" (UniqueName: \"kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt\") pod \"kuadrant-operator-controller-manager-86878c769f-jznk4\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.258572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.258548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.411772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.411745    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:09:29.412967 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:09:29.412941    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd702dad3_a57e_4a83_b891_626edfff2580.slice/crio-01420c6b83e69d09d5b807f03b28b3674a374b64f16820970b482823d72f0791 WatchSource:0}: Error finding container 01420c6b83e69d09d5b807f03b28b3674a374b64f16820970b482823d72f0791: Status 404 returned error can't find the container with id 01420c6b83e69d09d5b807f03b28b3674a374b64f16820970b482823d72f0791
Jul 09 04:09:29.727066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.726885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" event={"ID":"d702dad3-a57e-4a83-b891-626edfff2580","Type":"ContainerStarted","Data":"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"}
Jul 09 04:09:29.727066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.726946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" event={"ID":"d702dad3-a57e-4a83-b891-626edfff2580","Type":"ContainerStarted","Data":"01420c6b83e69d09d5b807f03b28b3674a374b64f16820970b482823d72f0791"}
Jul 09 04:09:29.727066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.727045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:29.728440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.728416    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" containerID="63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7" exitCode=0
Jul 09 04:09:29.728597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.728446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" event={"ID":"b7902334-164a-4b4c-9c6d-d0fce5a2271b","Type":"ContainerDied","Data":"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"}
Jul 09 04:09:29.728597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.728497    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"
Jul 09 04:09:29.728597 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.728511    2577 scope.go:117] "RemoveContainer" containerID="63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"
Jul 09 04:09:29.728765 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.728500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7" event={"ID":"b7902334-164a-4b4c-9c6d-d0fce5a2271b","Type":"ContainerDied","Data":"895982cb0d47a5e822329f41bb22971f4d5dc3f2cad07f291550ccd72e13a6de"}
Jul 09 04:09:29.742126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.741998    2577 scope.go:117] "RemoveContainer" containerID="63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"
Jul 09 04:09:29.742344 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:29.742309    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7\": container with ID starting with 63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7 not found: ID does not exist" containerID="63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"
Jul 09 04:09:29.742435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.742352    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7"} err="failed to get container status \"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7\": rpc error: code = NotFound desc = could not find container \"63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7\": container with ID starting with 63d054a89f4ee9ee2b7e525f5b3d90e4159224ab0ba7c6ee6ea44b463115a6d7 not found: ID does not exist"
Jul 09 04:09:29.748794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.748750    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" podStartSLOduration=1.748737471 podStartE2EDuration="1.748737471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:29.746217696 +0000 UTC m=+569.020595604" watchObservedRunningTime="2026-07-09 04:09:29.748737471 +0000 UTC m=+569.023115378"
Jul 09 04:09:29.759807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.759783    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:29.772697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:29.772677    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hshc7"]
Jul 09 04:09:31.180825 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:31.180797    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" path="/var/lib/kubelet/pods/b7902334-164a-4b4c-9c6d-d0fce5a2271b/volumes"
Jul 09 04:09:34.621349 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:34.621306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:34.720341 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:34.720305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:35.900391 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:35.900310    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:36.002340 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:36.002294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:40.737599 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:40.737570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:09:40.785037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:40.784988    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:40.785228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:40.785202    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" podUID="d845f4bd-f72d-49d0-9817-b42d6e674486" containerName="manager" containerID="cri-o://55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67" gracePeriod=10
Jul 09 04:09:41.040997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.040970    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:41.107935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.107904    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume\") pod \"d845f4bd-f72d-49d0-9817-b42d6e674486\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") "
Jul 09 04:09:41.108143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.107945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czpz5\" (UniqueName: \"kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5\") pod \"d845f4bd-f72d-49d0-9817-b42d6e674486\" (UID: \"d845f4bd-f72d-49d0-9817-b42d6e674486\") "
Jul 09 04:09:41.108291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.108267    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d845f4bd-f72d-49d0-9817-b42d6e674486" (UID: "d845f4bd-f72d-49d0-9817-b42d6e674486"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:09:41.110037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.110014    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5" (OuterVolumeSpecName: "kube-api-access-czpz5") pod "d845f4bd-f72d-49d0-9817-b42d6e674486" (UID: "d845f4bd-f72d-49d0-9817-b42d6e674486"). InnerVolumeSpecName "kube-api-access-czpz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:41.209316 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.209284    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d845f4bd-f72d-49d0-9817-b42d6e674486-extensions-socket-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:41.209316 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.209309    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czpz5\" (UniqueName: \"kubernetes.io/projected/d845f4bd-f72d-49d0-9817-b42d6e674486-kube-api-access-czpz5\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:09:41.800312 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.800274    2577 generic.go:356] "Generic (PLEG): container finished" podID="d845f4bd-f72d-49d0-9817-b42d6e674486" containerID="55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67" exitCode=0
Jul 09 04:09:41.800761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.800336    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"
Jul 09 04:09:41.800761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.800359    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" event={"ID":"d845f4bd-f72d-49d0-9817-b42d6e674486","Type":"ContainerDied","Data":"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"}
Jul 09 04:09:41.800761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.800426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4" event={"ID":"d845f4bd-f72d-49d0-9817-b42d6e674486","Type":"ContainerDied","Data":"360b78cfdfa866dbceb0154ce0e85ed6a4ac62994fcc7f0177381c98569ce3e9"}
Jul 09 04:09:41.800761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.800447    2577 scope.go:117] "RemoveContainer" containerID="55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"
Jul 09 04:09:41.809416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.809396    2577 scope.go:117] "RemoveContainer" containerID="55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"
Jul 09 04:09:41.809748 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:09:41.809729    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67\": container with ID starting with 55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67 not found: ID does not exist" containerID="55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"
Jul 09 04:09:41.809799 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.809758    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67"} err="failed to get container status \"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67\": rpc error: code = NotFound desc = could not find container \"55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67\": container with ID starting with 55000dfe5def4a5b25e4f4c41b619efa898b5c5c79b29fae59fad40100abeb67 not found: ID does not exist"
Jul 09 04:09:41.818708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.818676    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:41.826879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:41.826063    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hqhw4"]
Jul 09 04:09:43.180161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:43.180126    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d845f4bd-f72d-49d0-9817-b42d6e674486" path="/var/lib/kubelet/pods/d845f4bd-f72d-49d0-9817-b42d6e674486/volumes"
Jul 09 04:09:45.068102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:45.068071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:45.359814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:09:45.359777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:00.098541 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.098505    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:00.099037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.098925    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d845f4bd-f72d-49d0-9817-b42d6e674486" containerName="manager"
Jul 09 04:10:00.099037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.098938    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d845f4bd-f72d-49d0-9817-b42d6e674486" containerName="manager"
Jul 09 04:10:00.099037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.098956    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" containerName="manager"
Jul 09 04:10:00.099037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.098962    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" containerName="manager"
Jul 09 04:10:00.099037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.099035    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d845f4bd-f72d-49d0-9817-b42d6e674486" containerName="manager"
Jul 09 04:10:00.099265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.099048    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7902334-164a-4b4c-9c6d-d0fce5a2271b" containerName="manager"
Jul 09 04:10:00.100454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.100429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:00.102257 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.102242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.104206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.104184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lvckb\""
Jul 09 04:10:00.104206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.104184    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 04:10:00.109633 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.109609    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:00.172933 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.172903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9hk\" (UniqueName: \"kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.173102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.173041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.194822 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.194792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:00.202910 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.202871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:00.269163 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.269127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:00.274238 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.274220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9hk\" (UniqueName: \"kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.274372 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.274360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.275042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.275020    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.281790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.281763    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9hk\" (UniqueName: \"kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk\") pod \"limitador-limitador-64c94fd64f-lvxdq\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.414182 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.414082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:00.553791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.553760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:00.555388 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:00.555361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cdbfa4_a119_4891_86ce_3e5cd642ae0b.slice/crio-2d9cc454bf5a7f7f036a0f03c4f31d5cfaef773e4e4636ed4a7b8b289eb4a12a WatchSource:0}: Error finding container 2d9cc454bf5a7f7f036a0f03c4f31d5cfaef773e4e4636ed4a7b8b289eb4a12a: Status 404 returned error can't find the container with id 2d9cc454bf5a7f7f036a0f03c4f31d5cfaef773e4e4636ed4a7b8b289eb4a12a
Jul 09 04:10:00.885535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.885507    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:00.890172 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.890150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:00.892296 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.892261    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m285h\""
Jul 09 04:10:00.895425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.895402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:00.918082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.918043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" event={"ID":"91cdbfa4-a119-4891-86ce-3e5cd642ae0b","Type":"ContainerStarted","Data":"2d9cc454bf5a7f7f036a0f03c4f31d5cfaef773e4e4636ed4a7b8b289eb4a12a"}
Jul 09 04:10:00.923537 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.923496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:00.981088 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.981056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkwds\" (UniqueName: \"kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds\") pod \"authorino-58cd955675-k954r\" (UID: \"bccb49ce-36ee-4769-87ab-a7eb50782015\") " pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:00.985155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:00.985124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:01.079766 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.079738    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:10:01.080426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.080406    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:10:01.081522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.081500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkwds\" (UniqueName: \"kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds\") pod \"authorino-58cd955675-k954r\" (UID: \"bccb49ce-36ee-4769-87ab-a7eb50782015\") " pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:01.091888 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.091860    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkwds\" (UniqueName: \"kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds\") pod \"authorino-58cd955675-k954r\" (UID: \"bccb49ce-36ee-4769-87ab-a7eb50782015\") " pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:01.113592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.113571    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:01.117286 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.117269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:01.122958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.122931    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:01.182521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.182425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvhpg\" (UniqueName: \"kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg\") pod \"authorino-75d556bd77-g7tcz\" (UID: \"f08cbea0-1c98-46f2-8e28-d557736a3f82\") " pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:01.201997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.201969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m285h\""
Jul 09 04:10:01.211225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.211193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:01.283620 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.283558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvhpg\" (UniqueName: \"kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg\") pod \"authorino-75d556bd77-g7tcz\" (UID: \"f08cbea0-1c98-46f2-8e28-d557736a3f82\") " pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:01.291611 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.291581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvhpg\" (UniqueName: \"kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg\") pod \"authorino-75d556bd77-g7tcz\" (UID: \"f08cbea0-1c98-46f2-8e28-d557736a3f82\") " pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:01.384234 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.384206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:01.387047 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:01.387016    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbccb49ce_36ee_4769_87ab_a7eb50782015.slice/crio-7ff1f74d8d75cad69796bae6e43fd1376a2ff167dc38573b90e4d49ca91f56ad WatchSource:0}: Error finding container 7ff1f74d8d75cad69796bae6e43fd1376a2ff167dc38573b90e4d49ca91f56ad: Status 404 returned error can't find the container with id 7ff1f74d8d75cad69796bae6e43fd1376a2ff167dc38573b90e4d49ca91f56ad
Jul 09 04:10:01.428405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.428369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:01.602610 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.602574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:01.607863 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:01.607832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf08cbea0_1c98_46f2_8e28_d557736a3f82.slice/crio-cdcd2e961a20374c3b3cef0b28393c12223881dcd06dcc5f19c391d18f85139a WatchSource:0}: Error finding container cdcd2e961a20374c3b3cef0b28393c12223881dcd06dcc5f19c391d18f85139a: Status 404 returned error can't find the container with id cdcd2e961a20374c3b3cef0b28393c12223881dcd06dcc5f19c391d18f85139a
Jul 09 04:10:01.926941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.926908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k954r" event={"ID":"bccb49ce-36ee-4769-87ab-a7eb50782015","Type":"ContainerStarted","Data":"7ff1f74d8d75cad69796bae6e43fd1376a2ff167dc38573b90e4d49ca91f56ad"}
Jul 09 04:10:01.928105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:01.928073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g7tcz" event={"ID":"f08cbea0-1c98-46f2-8e28-d557736a3f82","Type":"ContainerStarted","Data":"cdcd2e961a20374c3b3cef0b28393c12223881dcd06dcc5f19c391d18f85139a"}
Jul 09 04:10:06.968772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.968735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" event={"ID":"91cdbfa4-a119-4891-86ce-3e5cd642ae0b","Type":"ContainerStarted","Data":"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"}
Jul 09 04:10:06.969178 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.968830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:06.970290 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.970265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g7tcz" event={"ID":"f08cbea0-1c98-46f2-8e28-d557736a3f82","Type":"ContainerStarted","Data":"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"}
Jul 09 04:10:06.971712 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.971682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k954r" event={"ID":"bccb49ce-36ee-4769-87ab-a7eb50782015","Type":"ContainerStarted","Data":"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"}
Jul 09 04:10:06.983808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.983765    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" podStartSLOduration=1.544025908 podStartE2EDuration="6.983749711s" totalImagesPullingTime="5.439723803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:06.983309079 +0000 UTC m=+606.257686999" watchObservedRunningTime="2026-07-09 04:10:06.983749711 +0000 UTC m=+606.258127618"
Jul 09 04:10:06.996641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:06.996602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k954r" podStartSLOduration=2.400183255 podStartE2EDuration="6.996591097s" totalImagesPullingTime="4.596407842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:06.995014066 +0000 UTC m=+606.269391989" watchObservedRunningTime="2026-07-09 04:10:06.996591097 +0000 UTC m=+606.270969005"
Jul 09 04:10:07.013942 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:07.013904    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-g7tcz" podStartSLOduration=1.637905284 podStartE2EDuration="6.01389187s" totalImagesPullingTime="4.375986586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:07.012061037 +0000 UTC m=+606.286438970" watchObservedRunningTime="2026-07-09 04:10:07.01389187 +0000 UTC m=+606.288269778"
Jul 09 04:10:07.039777 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:07.039751    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:08.982239 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:08.982204    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k954r" podUID="bccb49ce-36ee-4769-87ab-a7eb50782015" containerName="authorino" containerID="cri-o://f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d" gracePeriod=30
Jul 09 04:10:09.227492 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.227449    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:09.364350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.364311    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkwds\" (UniqueName: \"kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds\") pod \"bccb49ce-36ee-4769-87ab-a7eb50782015\" (UID: \"bccb49ce-36ee-4769-87ab-a7eb50782015\") "
Jul 09 04:10:09.366267 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.366241    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds" (OuterVolumeSpecName: "kube-api-access-kkwds") pod "bccb49ce-36ee-4769-87ab-a7eb50782015" (UID: "bccb49ce-36ee-4769-87ab-a7eb50782015"). InnerVolumeSpecName "kube-api-access-kkwds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:10:09.465441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.465414    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkwds\" (UniqueName: \"kubernetes.io/projected/bccb49ce-36ee-4769-87ab-a7eb50782015-kube-api-access-kkwds\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:09.989183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.989153    2577 generic.go:356] "Generic (PLEG): container finished" podID="bccb49ce-36ee-4769-87ab-a7eb50782015" containerID="f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d" exitCode=0
Jul 09 04:10:09.989637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.989188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k954r" event={"ID":"bccb49ce-36ee-4769-87ab-a7eb50782015","Type":"ContainerDied","Data":"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"}
Jul 09 04:10:09.989637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.989197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k954r"
Jul 09 04:10:09.989637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.989214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k954r" event={"ID":"bccb49ce-36ee-4769-87ab-a7eb50782015","Type":"ContainerDied","Data":"7ff1f74d8d75cad69796bae6e43fd1376a2ff167dc38573b90e4d49ca91f56ad"}
Jul 09 04:10:09.989637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.989231    2577 scope.go:117] "RemoveContainer" containerID="f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"
Jul 09 04:10:09.998832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.998813    2577 scope.go:117] "RemoveContainer" containerID="f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"
Jul 09 04:10:09.999097 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:09.999078    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d\": container with ID starting with f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d not found: ID does not exist" containerID="f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"
Jul 09 04:10:09.999158 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:09.999105    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d"} err="failed to get container status \"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d\": rpc error: code = NotFound desc = could not find container \"f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d\": container with ID starting with f37be116c064e08776a9ae60106155462f4fa52b069eb4e71523ee2dc4a2713d not found: ID does not exist"
Jul 09 04:10:10.008216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:10.008192    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:10.011821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:10.011801    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k954r"]
Jul 09 04:10:11.180228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:11.180190    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bccb49ce-36ee-4769-87ab-a7eb50782015" path="/var/lib/kubelet/pods/bccb49ce-36ee-4769-87ab-a7eb50782015/volumes"
Jul 09 04:10:15.781892 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:15.781849    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:15.784408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:15.782227    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" podUID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" containerName="limitador" containerID="cri-o://84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf" gracePeriod=30
Jul 09 04:10:15.784408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:15.784138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:16.335482 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.335453    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:16.423861 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.423834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file\") pod \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") "
Jul 09 04:10:16.424051 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.423920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk9hk\" (UniqueName: \"kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk\") pod \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\" (UID: \"91cdbfa4-a119-4891-86ce-3e5cd642ae0b\") "
Jul 09 04:10:16.424200 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.424175    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file" (OuterVolumeSpecName: "config-file") pod "91cdbfa4-a119-4891-86ce-3e5cd642ae0b" (UID: "91cdbfa4-a119-4891-86ce-3e5cd642ae0b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:10:16.425934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.425911    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk" (OuterVolumeSpecName: "kube-api-access-vk9hk") pod "91cdbfa4-a119-4891-86ce-3e5cd642ae0b" (UID: "91cdbfa4-a119-4891-86ce-3e5cd642ae0b"). InnerVolumeSpecName "kube-api-access-vk9hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:10:16.524844 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.524809    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-config-file\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:16.524844 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:16.524842    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk9hk\" (UniqueName: \"kubernetes.io/projected/91cdbfa4-a119-4891-86ce-3e5cd642ae0b-kube-api-access-vk9hk\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:17.034429 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.034397    2577 generic.go:356] "Generic (PLEG): container finished" podID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" containerID="84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf" exitCode=0
Jul 09 04:10:17.034899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.034444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" event={"ID":"91cdbfa4-a119-4891-86ce-3e5cd642ae0b","Type":"ContainerDied","Data":"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"}
Jul 09 04:10:17.034899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.034450    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"
Jul 09 04:10:17.034899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.034528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lvxdq" event={"ID":"91cdbfa4-a119-4891-86ce-3e5cd642ae0b","Type":"ContainerDied","Data":"2d9cc454bf5a7f7f036a0f03c4f31d5cfaef773e4e4636ed4a7b8b289eb4a12a"}
Jul 09 04:10:17.034899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.034551    2577 scope.go:117] "RemoveContainer" containerID="84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"
Jul 09 04:10:17.043865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.043846    2577 scope.go:117] "RemoveContainer" containerID="84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"
Jul 09 04:10:17.044129 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:17.044109    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf\": container with ID starting with 84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf not found: ID does not exist" containerID="84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"
Jul 09 04:10:17.044190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.044141    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf"} err="failed to get container status \"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf\": rpc error: code = NotFound desc = could not find container \"84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf\": container with ID starting with 84aaf6f503743eab0bba1d3b14cc4145ccd4aae8ade2762bd1af63d1597f4bdf not found: ID does not exist"
Jul 09 04:10:17.057441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.057421    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:17.062759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.062735    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lvxdq"]
Jul 09 04:10:17.180244 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:17.180213    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" path="/var/lib/kubelet/pods/91cdbfa4-a119-4891-86ce-3e5cd642ae0b/volumes"
Jul 09 04:10:18.378445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:18.378421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:18.475872 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:18.475842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:18.831260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:18.831175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:30.438118 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.438034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:30.674977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.674940    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:30.675335 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675323    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" containerName="limitador"
Jul 09 04:10:30.675380 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675337    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" containerName="limitador"
Jul 09 04:10:30.675380 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675358    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccb49ce-36ee-4769-87ab-a7eb50782015" containerName="authorino"
Jul 09 04:10:30.675380 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675363    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccb49ce-36ee-4769-87ab-a7eb50782015" containerName="authorino"
Jul 09 04:10:30.675504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675432    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccb49ce-36ee-4769-87ab-a7eb50782015" containerName="authorino"
Jul 09 04:10:30.675504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.675441    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cdbfa4-a119-4891-86ce-3e5cd642ae0b" containerName="limitador"
Jul 09 04:10:30.681832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.681816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:30.695826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.695776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:30.716905 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.716876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:30.754409 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.754381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz42f\" (UniqueName: \"kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f\") pod \"authorino-5d4b67c446-sh2b5\" (UID: \"779fb894-d899-436f-8a61-28472a8c4629\") " pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:30.854837 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.854805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz42f\" (UniqueName: \"kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f\") pod \"authorino-5d4b67c446-sh2b5\" (UID: \"779fb894-d899-436f-8a61-28472a8c4629\") " pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:30.864227 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.864201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz42f\" (UniqueName: \"kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f\") pod \"authorino-5d4b67c446-sh2b5\" (UID: \"779fb894-d899-436f-8a61-28472a8c4629\") " pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:30.899390 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.899361    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:30.899645 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.899633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:30.926148 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.926121    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:10:30.932120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.932099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:30.934121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.934098    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 04:10:30.936584 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.936561    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:10:30.955535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.955411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kqjx\" (UniqueName: \"kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:30.955535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:30.955477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.037959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.037931    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:31.040055 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:31.040027    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod779fb894_d899_436f_8a61_28472a8c4629.slice/crio-89a2f7fd5cb79ec9c3f7eb3141598839e9b17c60a55ca975bee45893f95b5a91 WatchSource:0}: Error finding container 89a2f7fd5cb79ec9c3f7eb3141598839e9b17c60a55ca975bee45893f95b5a91: Status 404 returned error can't find the container with id 89a2f7fd5cb79ec9c3f7eb3141598839e9b17c60a55ca975bee45893f95b5a91
Jul 09 04:10:31.056298 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.056275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kqjx\" (UniqueName: \"kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.056414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.056311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.058599 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.058581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.063225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.063197    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kqjx\" (UniqueName: \"kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx\") pod \"authorino-54b988b6cc-x47pq\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") " pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.109999 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.109966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" event={"ID":"779fb894-d899-436f-8a61-28472a8c4629","Type":"ContainerStarted","Data":"89a2f7fd5cb79ec9c3f7eb3141598839e9b17c60a55ca975bee45893f95b5a91"}
Jul 09 04:10:31.243892 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.243821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:10:31.376598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:31.376576    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:10:31.378398 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:31.378370    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbe53063_fbfe_40e4_9489_0095c64495d1.slice/crio-9f744d9981172c31277d7c9567942642310c1a278b145efcf676ff4ad68e2dcb WatchSource:0}: Error finding container 9f744d9981172c31277d7c9567942642310c1a278b145efcf676ff4ad68e2dcb: Status 404 returned error can't find the container with id 9f744d9981172c31277d7c9567942642310c1a278b145efcf676ff4ad68e2dcb
Jul 09 04:10:32.116898 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.116859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b988b6cc-x47pq" event={"ID":"cbe53063-fbfe-40e4-9489-0095c64495d1","Type":"ContainerStarted","Data":"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"}
Jul 09 04:10:32.116898 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.116902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b988b6cc-x47pq" event={"ID":"cbe53063-fbfe-40e4-9489-0095c64495d1","Type":"ContainerStarted","Data":"9f744d9981172c31277d7c9567942642310c1a278b145efcf676ff4ad68e2dcb"}
Jul 09 04:10:32.118228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.118208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" event={"ID":"779fb894-d899-436f-8a61-28472a8c4629","Type":"ContainerStarted","Data":"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"}
Jul 09 04:10:32.118341 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.118298    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" podUID="779fb894-d899-436f-8a61-28472a8c4629" containerName="authorino" containerID="cri-o://72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8" gracePeriod=30
Jul 09 04:10:32.130629 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.130590    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54b988b6cc-x47pq" podStartSLOduration=1.570747246 podStartE2EDuration="2.130577151s" totalImagesPullingTime="559.829905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:32.129138454 +0000 UTC m=+631.403516363" watchObservedRunningTime="2026-07-09 04:10:32.130577151 +0000 UTC m=+631.404955058"
Jul 09 04:10:32.142600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.142550    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" podStartSLOduration=1.759805361 podStartE2EDuration="2.142530376s" totalImagesPullingTime="382.725015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:32.141158409 +0000 UTC m=+631.415536317" watchObservedRunningTime="2026-07-09 04:10:32.142530376 +0000 UTC m=+631.416908285"
Jul 09 04:10:32.157053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.157026    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:32.157187 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.157169    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-g7tcz" podUID="f08cbea0-1c98-46f2-8e28-d557736a3f82" containerName="authorino" containerID="cri-o://646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13" gracePeriod=30
Jul 09 04:10:32.409375 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.409355    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:32.414896 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.414875    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:32.468739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.468705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz42f\" (UniqueName: \"kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f\") pod \"779fb894-d899-436f-8a61-28472a8c4629\" (UID: \"779fb894-d899-436f-8a61-28472a8c4629\") "
Jul 09 04:10:32.468901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.468786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvhpg\" (UniqueName: \"kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg\") pod \"f08cbea0-1c98-46f2-8e28-d557736a3f82\" (UID: \"f08cbea0-1c98-46f2-8e28-d557736a3f82\") "
Jul 09 04:10:32.470782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.470752    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f" (OuterVolumeSpecName: "kube-api-access-fz42f") pod "779fb894-d899-436f-8a61-28472a8c4629" (UID: "779fb894-d899-436f-8a61-28472a8c4629"). InnerVolumeSpecName "kube-api-access-fz42f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:10:32.470782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.470777    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg" (OuterVolumeSpecName: "kube-api-access-gvhpg") pod "f08cbea0-1c98-46f2-8e28-d557736a3f82" (UID: "f08cbea0-1c98-46f2-8e28-d557736a3f82"). InnerVolumeSpecName "kube-api-access-gvhpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:10:32.569927 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.569841    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz42f\" (UniqueName: \"kubernetes.io/projected/779fb894-d899-436f-8a61-28472a8c4629-kube-api-access-fz42f\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:32.569927 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:32.569872    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvhpg\" (UniqueName: \"kubernetes.io/projected/f08cbea0-1c98-46f2-8e28-d557736a3f82-kube-api-access-gvhpg\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:33.124851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.124759    2577 generic.go:356] "Generic (PLEG): container finished" podID="779fb894-d899-436f-8a61-28472a8c4629" containerID="72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8" exitCode=0
Jul 09 04:10:33.124851 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.124809    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sh2b5"
Jul 09 04:10:33.125304 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.124844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" event={"ID":"779fb894-d899-436f-8a61-28472a8c4629","Type":"ContainerDied","Data":"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"}
Jul 09 04:10:33.125304 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.124883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sh2b5" event={"ID":"779fb894-d899-436f-8a61-28472a8c4629","Type":"ContainerDied","Data":"89a2f7fd5cb79ec9c3f7eb3141598839e9b17c60a55ca975bee45893f95b5a91"}
Jul 09 04:10:33.125304 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.124901    2577 scope.go:117] "RemoveContainer" containerID="72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"
Jul 09 04:10:33.126307 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.126283    2577 generic.go:356] "Generic (PLEG): container finished" podID="f08cbea0-1c98-46f2-8e28-d557736a3f82" containerID="646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13" exitCode=0
Jul 09 04:10:33.126383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.126345    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g7tcz"
Jul 09 04:10:33.126383 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.126365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g7tcz" event={"ID":"f08cbea0-1c98-46f2-8e28-d557736a3f82","Type":"ContainerDied","Data":"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"}
Jul 09 04:10:33.126492 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.126399    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g7tcz" event={"ID":"f08cbea0-1c98-46f2-8e28-d557736a3f82","Type":"ContainerDied","Data":"cdcd2e961a20374c3b3cef0b28393c12223881dcd06dcc5f19c391d18f85139a"}
Jul 09 04:10:33.135771 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.135750    2577 scope.go:117] "RemoveContainer" containerID="72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"
Jul 09 04:10:33.136061 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:33.136034    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8\": container with ID starting with 72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8 not found: ID does not exist" containerID="72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"
Jul 09 04:10:33.136190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.136072    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8"} err="failed to get container status \"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8\": rpc error: code = NotFound desc = could not find container \"72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8\": container with ID starting with 72ceaf3cf4ea096361f883b6ec9fcb1d53a650cf28d1f93e05ce2bd7cb4339a8 not found: ID does not exist"
Jul 09 04:10:33.136190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.136094    2577 scope.go:117] "RemoveContainer" containerID="646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"
Jul 09 04:10:33.143874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.143860    2577 scope.go:117] "RemoveContainer" containerID="646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"
Jul 09 04:10:33.144122 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:33.144106    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13\": container with ID starting with 646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13 not found: ID does not exist" containerID="646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"
Jul 09 04:10:33.144161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.144130    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13"} err="failed to get container status \"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13\": rpc error: code = NotFound desc = could not find container \"646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13\": container with ID starting with 646b425712ece1d7c83f38a605974def0195731338d0cbf7d96ab6444c525b13 not found: ID does not exist"
Jul 09 04:10:33.151235 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.151216    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:33.156967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.156948    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sh2b5"]
Jul 09 04:10:33.165237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.165218    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:33.168300 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.168275    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g7tcz"]
Jul 09 04:10:33.179173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.179154    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779fb894-d899-436f-8a61-28472a8c4629" path="/var/lib/kubelet/pods/779fb894-d899-436f-8a61-28472a8c4629/volumes"
Jul 09 04:10:33.179456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:33.179445    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f08cbea0-1c98-46f2-8e28-d557736a3f82" path="/var/lib/kubelet/pods/f08cbea0-1c98-46f2-8e28-d557736a3f82/volumes"
Jul 09 04:10:36.233839 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.233808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:36.258587 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.258560    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:36.259042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259024    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779fb894-d899-436f-8a61-28472a8c4629" containerName="authorino"
Jul 09 04:10:36.259042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259041    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="779fb894-d899-436f-8a61-28472a8c4629" containerName="authorino"
Jul 09 04:10:36.259225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259056    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08cbea0-1c98-46f2-8e28-d557736a3f82" containerName="authorino"
Jul 09 04:10:36.259225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259064    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08cbea0-1c98-46f2-8e28-d557736a3f82" containerName="authorino"
Jul 09 04:10:36.259225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259163    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08cbea0-1c98-46f2-8e28-d557736a3f82" containerName="authorino"
Jul 09 04:10:36.259225 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.259173    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="779fb894-d899-436f-8a61-28472a8c4629" containerName="authorino"
Jul 09 04:10:36.333428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.333405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:36.333585 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.333507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.335164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.335139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:36.335343 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.335326    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 04:10:36.335404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.335332    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 04:10:36.335404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.335370    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-thkpz\""
Jul 09 04:10:36.395559 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.395525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:36.403126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.403106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.403222 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.403140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9sdf\" (UniqueName: \"kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.431227 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.431202    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:10:36.458709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.458682    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:10:36.458826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.458782    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.504344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.504290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.504344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.504318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.504344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.504343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9sdf\" (UniqueName: \"kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.504556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.504406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d598t\" (UniqueName: \"kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.504556 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:36.504453    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 09 04:10:36.504632 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:36.504560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert podName:91c609ec-fbec-4458-b5f4-0498f06c9749 nodeName:}" failed. No retries permitted until 2026-07-09 04:10:37.004545591 +0000 UTC m=+636.278923476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert") pod "maas-controller-5577f956cc-rdgz2" (UID: "91c609ec-fbec-4458-b5f4-0498f06c9749") : secret "maas-controller-webhook-cert" not found
Jul 09 04:10:36.512421 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.512404    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9sdf\" (UniqueName: \"kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:36.574373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.574338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:36.605172 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.605148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d598t\" (UniqueName: \"kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.605272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.605207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.607510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.607494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.612902 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.612881    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d598t\" (UniqueName: \"kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t\") pod \"maas-controller-68d56bd64d-czdbs\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") " pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.768534 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.768453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:36.911723 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:36.911702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:10:36.913278 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:36.913255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cda9ee4_71b2_468a_841d_bb9416042098.slice/crio-833a9cfaac8f8c365ff84c3366cbfee5c17c1ededcabb8c3064c36ffba585684 WatchSource:0}: Error finding container 833a9cfaac8f8c365ff84c3366cbfee5c17c1ededcabb8c3064c36ffba585684: Status 404 returned error can't find the container with id 833a9cfaac8f8c365ff84c3366cbfee5c17c1ededcabb8c3064c36ffba585684
Jul 09 04:10:37.009698 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:37.009673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:37.011859 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:37.011839    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") pod \"maas-controller-5577f956cc-rdgz2\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") " pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:37.149459 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:37.149430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-czdbs" event={"ID":"6cda9ee4-71b2-468a-841d-bb9416042098","Type":"ContainerStarted","Data":"833a9cfaac8f8c365ff84c3366cbfee5c17c1ededcabb8c3064c36ffba585684"}
Jul 09 04:10:37.244447 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:37.244416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:37.403542 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:37.403518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:37.405346 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:10:37.405321    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c609ec_fbec_4458_b5f4_0498f06c9749.slice/crio-655178d204f5c7d576f5efa7b39a70ecc651525352b5977afa8fd55d840f1035 WatchSource:0}: Error finding container 655178d204f5c7d576f5efa7b39a70ecc651525352b5977afa8fd55d840f1035: Status 404 returned error can't find the container with id 655178d204f5c7d576f5efa7b39a70ecc651525352b5977afa8fd55d840f1035
Jul 09 04:10:38.161361 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:38.161319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5577f956cc-rdgz2" event={"ID":"91c609ec-fbec-4458-b5f4-0498f06c9749","Type":"ContainerStarted","Data":"655178d204f5c7d576f5efa7b39a70ecc651525352b5977afa8fd55d840f1035"}
Jul 09 04:10:39.816048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:39.816012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:39.911652 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:39.911562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:40.174059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.173965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5577f956cc-rdgz2" event={"ID":"91c609ec-fbec-4458-b5f4-0498f06c9749","Type":"ContainerStarted","Data":"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"}
Jul 09 04:10:40.174059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.174056    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:40.175587 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.175559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-czdbs" event={"ID":"6cda9ee4-71b2-468a-841d-bb9416042098","Type":"ContainerStarted","Data":"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"}
Jul 09 04:10:40.175703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.175636    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:40.190771 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.190726    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5577f956cc-rdgz2" podStartSLOduration=1.98821056 podStartE2EDuration="4.190710903s" totalImagesPullingTime="2.202500343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:40.187829306 +0000 UTC m=+639.462207248" watchObservedRunningTime="2026-07-09 04:10:40.190710903 +0000 UTC m=+639.465088810"
Jul 09 04:10:40.217542 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:40.217495    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68d56bd64d-czdbs" podStartSLOduration=1.511629197 podStartE2EDuration="4.217480042s" totalImagesPullingTime="2.705850845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:40.213587319 +0000 UTC m=+639.487965238" watchObservedRunningTime="2026-07-09 04:10:40.217480042 +0000 UTC m=+639.491858153"
Jul 09 04:10:51.186039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.186013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:10:51.186402 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.186072    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:51.243986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.243952    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:51.244206 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.244180    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5577f956cc-rdgz2" podUID="91c609ec-fbec-4458-b5f4-0498f06c9749" containerName="manager" containerID="cri-o://b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da" gracePeriod=30
Jul 09 04:10:51.486782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.486757    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:51.659895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.659866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") pod \"91c609ec-fbec-4458-b5f4-0498f06c9749\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") "
Jul 09 04:10:51.660095 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.659932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9sdf\" (UniqueName: \"kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf\") pod \"91c609ec-fbec-4458-b5f4-0498f06c9749\" (UID: \"91c609ec-fbec-4458-b5f4-0498f06c9749\") "
Jul 09 04:10:51.662068 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.662039    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "91c609ec-fbec-4458-b5f4-0498f06c9749" (UID: "91c609ec-fbec-4458-b5f4-0498f06c9749"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:10:51.662163 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.662062    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf" (OuterVolumeSpecName: "kube-api-access-b9sdf") pod "91c609ec-fbec-4458-b5f4-0498f06c9749" (UID: "91c609ec-fbec-4458-b5f4-0498f06c9749"). InnerVolumeSpecName "kube-api-access-b9sdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:10:51.761529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.761429    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91c609ec-fbec-4458-b5f4-0498f06c9749-webhook-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:51.761529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:51.761460    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9sdf\" (UniqueName: \"kubernetes.io/projected/91c609ec-fbec-4458-b5f4-0498f06c9749-kube-api-access-b9sdf\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:10:52.239173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.239136    2577 generic.go:356] "Generic (PLEG): container finished" podID="91c609ec-fbec-4458-b5f4-0498f06c9749" containerID="b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da" exitCode=0
Jul 09 04:10:52.239646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.239181    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5577f956cc-rdgz2"
Jul 09 04:10:52.239646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.239222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5577f956cc-rdgz2" event={"ID":"91c609ec-fbec-4458-b5f4-0498f06c9749","Type":"ContainerDied","Data":"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"}
Jul 09 04:10:52.239646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.239277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5577f956cc-rdgz2" event={"ID":"91c609ec-fbec-4458-b5f4-0498f06c9749","Type":"ContainerDied","Data":"655178d204f5c7d576f5efa7b39a70ecc651525352b5977afa8fd55d840f1035"}
Jul 09 04:10:52.239646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.239299    2577 scope.go:117] "RemoveContainer" containerID="b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"
Jul 09 04:10:52.248663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.248428    2577 scope.go:117] "RemoveContainer" containerID="b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"
Jul 09 04:10:52.248715 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:10:52.248661    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da\": container with ID starting with b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da not found: ID does not exist" containerID="b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"
Jul 09 04:10:52.248715 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.248684    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da"} err="failed to get container status \"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da\": rpc error: code = NotFound desc = could not find container \"b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da\": container with ID starting with b2087b1e6f3d870c3ff692167630c7243536d8a39ab7cef3452a308e53bc22da not found: ID does not exist"
Jul 09 04:10:52.259201 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.259179    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:52.268625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.263207    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5577f956cc-rdgz2"]
Jul 09 04:10:52.868119 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:52.868073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:53.180455 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:10:53.180368    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c609ec-fbec-4458-b5f4-0498f06c9749" path="/var/lib/kubelet/pods/91c609ec-fbec-4458-b5f4-0498f06c9749/volumes"
Jul 09 04:11:10.748034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:10.747996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:11.226148 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.226116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:11.263770 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.263741    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"]
Jul 09 04:11:11.264184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.264163    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c609ec-fbec-4458-b5f4-0498f06c9749" containerName="manager"
Jul 09 04:11:11.264184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.264184    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c609ec-fbec-4458-b5f4-0498f06c9749" containerName="manager"
Jul 09 04:11:11.264369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.264289    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c609ec-fbec-4458-b5f4-0498f06c9749" containerName="manager"
Jul 09 04:11:11.268796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.268777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.270674 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.270643    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:11:11.270674 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.270665    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 04:11:11.270856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.270678    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:11:11.270856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.270781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svwzx\""
Jul 09 04:11:11.275865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.275839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"]
Jul 09 04:11:11.319249 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.319219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rww\" (UniqueName: \"kubernetes.io/projected/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-kube-api-access-t2rww\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.319396 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.319270    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.319396 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.319293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-openshift-service-ca\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.420039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.420015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rww\" (UniqueName: \"kubernetes.io/projected/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-kube-api-access-t2rww\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.420213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.420067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.420213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.420090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-openshift-service-ca\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.420213 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:11:11.420167    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 09 04:11:11.420397 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:11:11.420237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls podName:e6afaf93-9b8f-46e5-9711-7b4bc8c000d5 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:11.92022142 +0000 UTC m=+671.194599306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls") pod "maas-api-6dc7d55d4-px48k" (UID: "e6afaf93-9b8f-46e5-9711-7b4bc8c000d5") : secret "maas-api-serving-cert" not found
Jul 09 04:11:11.420740 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.420723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-openshift-service-ca\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.431938 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.431919    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rww\" (UniqueName: \"kubernetes.io/projected/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-kube-api-access-t2rww\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.644808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.644776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:11.748325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.748272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:11.755377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.755350    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8fq2v"]
Jul 09 04:11:11.759101 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.759084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.761369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.761346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hsms2\""
Jul 09 04:11:11.761506 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.761418    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 04:11:11.769166 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.769142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8fq2v"]
Jul 09 04:11:11.794880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.794845    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nmzr6"]
Jul 09 04:11:11.798796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.798766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.807552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.807533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nmzr6"]
Jul 09 04:11:11.824013 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.823988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/416930c5-8930-4dcf-b5ce-ec9154266a5e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.824131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.824042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cce0859-a3ef-4fb4-8742-5d079b312cd1-plugins-config-volume\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.824131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.824075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vcmn\" (UniqueName: \"kubernetes.io/projected/416930c5-8930-4dcf-b5ce-ec9154266a5e-kube-api-access-9vcmn\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.824131 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.824113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqc44\" (UniqueName: \"kubernetes.io/projected/1cce0859-a3ef-4fb4-8742-5d079b312cd1-kube-api-access-gqc44\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.854808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.854780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:11.924900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.924817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cce0859-a3ef-4fb4-8742-5d079b312cd1-plugins-config-volume\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.924900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.924856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vcmn\" (UniqueName: \"kubernetes.io/projected/416930c5-8930-4dcf-b5ce-ec9154266a5e-kube-api-access-9vcmn\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.925124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.924993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqc44\" (UniqueName: \"kubernetes.io/projected/1cce0859-a3ef-4fb4-8742-5d079b312cd1-kube-api-access-gqc44\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.925124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.925099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/416930c5-8930-4dcf-b5ce-ec9154266a5e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.925238 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.925159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.925695 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.925671    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cce0859-a3ef-4fb4-8742-5d079b312cd1-plugins-config-volume\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.925920 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.925901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/416930c5-8930-4dcf-b5ce-ec9154266a5e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:11.927867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.927843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6afaf93-9b8f-46e5-9711-7b4bc8c000d5-maas-api-tls\") pod \"maas-api-6dc7d55d4-px48k\" (UID: \"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5\") " pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:11.931821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.931801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqc44\" (UniqueName: \"kubernetes.io/projected/1cce0859-a3ef-4fb4-8742-5d079b312cd1-kube-api-access-gqc44\") pod \"payload-processing-58f86c7798-nmzr6\" (UID: \"1cce0859-a3ef-4fb4-8742-5d079b312cd1\") " pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:11.932395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:11.932367    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vcmn\" (UniqueName: \"kubernetes.io/projected/416930c5-8930-4dcf-b5ce-ec9154266a5e-kube-api-access-9vcmn\") pod \"payload-pre-processing-7964457696-8fq2v\" (UID: \"416930c5-8930-4dcf-b5ce-ec9154266a5e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:12.070806 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.070778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:12.110258 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.110225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:12.182428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.182251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:12.224145 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.223859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8fq2v"]
Jul 09 04:11:12.228684 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:11:12.228635    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416930c5_8930_4dcf_b5ce_ec9154266a5e.slice/crio-d371d77420b8cc247d4663ff72e87aeb5217fe9fbfc093c7a51e0b1cae5f02a4 WatchSource:0}: Error finding container d371d77420b8cc247d4663ff72e87aeb5217fe9fbfc093c7a51e0b1cae5f02a4: Status 404 returned error can't find the container with id d371d77420b8cc247d4663ff72e87aeb5217fe9fbfc093c7a51e0b1cae5f02a4
Jul 09 04:11:12.231296 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.231274    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:11:12.283366 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.283337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nmzr6"]
Jul 09 04:11:12.285537 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:11:12.285438    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cce0859_a3ef_4fb4_8742_5d079b312cd1.slice/crio-818ef604477a9a31b30ea465657ab02e79aaa61be6724e06ae9e23f7057f294f WatchSource:0}: Error finding container 818ef604477a9a31b30ea465657ab02e79aaa61be6724e06ae9e23f7057f294f: Status 404 returned error can't find the container with id 818ef604477a9a31b30ea465657ab02e79aaa61be6724e06ae9e23f7057f294f
Jul 09 04:11:12.338509 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.338462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"]
Jul 09 04:11:12.340012 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:11:12.339981    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6afaf93_9b8f_46e5_9711_7b4bc8c000d5.slice/crio-27cf9660598d35750a64a9ae9bb0387024d31d995e8fadecba52380cc6014449 WatchSource:0}: Error finding container 27cf9660598d35750a64a9ae9bb0387024d31d995e8fadecba52380cc6014449: Status 404 returned error can't find the container with id 27cf9660598d35750a64a9ae9bb0387024d31d995e8fadecba52380cc6014449
Jul 09 04:11:12.352295 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.352261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k" event={"ID":"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5","Type":"ContainerStarted","Data":"27cf9660598d35750a64a9ae9bb0387024d31d995e8fadecba52380cc6014449"}
Jul 09 04:11:12.353649 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.353622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6" event={"ID":"1cce0859-a3ef-4fb4-8742-5d079b312cd1","Type":"ContainerStarted","Data":"818ef604477a9a31b30ea465657ab02e79aaa61be6724e06ae9e23f7057f294f"}
Jul 09 04:11:12.354832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:12.354810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v" event={"ID":"416930c5-8930-4dcf-b5ce-ec9154266a5e","Type":"ContainerStarted","Data":"d371d77420b8cc247d4663ff72e87aeb5217fe9fbfc093c7a51e0b1cae5f02a4"}
Jul 09 04:11:14.370281 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.370191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k" event={"ID":"e6afaf93-9b8f-46e5-9711-7b4bc8c000d5","Type":"ContainerStarted","Data":"ee22c6a8e8b2848bc2f5f210e8de13cd2d9598da1c1bbb92118a63327dc61a0d"}
Jul 09 04:11:14.370281 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.370263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:14.371829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.371802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6" event={"ID":"1cce0859-a3ef-4fb4-8742-5d079b312cd1","Type":"ContainerStarted","Data":"c51b012d312ee0d51e4ceab1f60c246048ddb28be74fe6703d8936c69d201866"}
Jul 09 04:11:14.371981 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.371861    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:14.373568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.373537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v" event={"ID":"416930c5-8930-4dcf-b5ce-ec9154266a5e","Type":"ContainerStarted","Data":"ff005ddafc10b3abdb72fb648d557c781356177e40a242374aabdbaa075288ee"}
Jul 09 04:11:14.373691 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.373620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:14.386377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.386337    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k" podStartSLOduration=1.627098181 podStartE2EDuration="3.38632524s" totalImagesPullingTime="1.759227059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:14.384597451 +0000 UTC m=+673.658975383" watchObservedRunningTime="2026-07-09 04:11:14.38632524 +0000 UTC m=+673.660703148"
Jul 09 04:11:14.398173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.398136    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6" podStartSLOduration=2.048519103 podStartE2EDuration="3.398123938s" totalImagesPullingTime="1.349604835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:14.396942643 +0000 UTC m=+673.671320569" watchObservedRunningTime="2026-07-09 04:11:14.398123938 +0000 UTC m=+673.672501823"
Jul 09 04:11:14.410251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:14.410209    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v" podStartSLOduration=2.002006918 podStartE2EDuration="3.410198959s" totalImagesPullingTime="1.408192041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:14.409402351 +0000 UTC m=+673.683780259" watchObservedRunningTime="2026-07-09 04:11:14.410198959 +0000 UTC m=+673.684576866"
Jul 09 04:11:20.384402 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:20.384375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6dc7d55d4-px48k"
Jul 09 04:11:21.355636 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:21.355600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:21.458056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:21.458027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:23.540964 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:23.540928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:23.641749 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:23.641719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:25.381638 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:25.381611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-nmzr6"
Jul 09 04:11:25.382116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:25.381989    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-8fq2v"
Jul 09 04:11:31.323104 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:31.323066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:36.582372 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.582342    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"]
Jul 09 04:11:36.586274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.586252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.587854 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.587833    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 04:11:36.588041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.588030    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 04:11:36.588253 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.588233    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 04:11:36.588409 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.588398    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-trvcm\""
Jul 09 04:11:36.592962 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.592941    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"]
Jul 09 04:11:36.642560 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.642738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.642738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.642738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.642738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fg7z\" (UniqueName: \"kubernetes.io/projected/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-kube-api-access-8fg7z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.643069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.642801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.689315 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.689278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:36.743363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fg7z\" (UniqueName: \"kubernetes.io/projected/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-kube-api-access-8fg7z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.743911 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.744038 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.743939    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.744102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.744084    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.745678 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.745660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.745967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.745951    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.750617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.750596    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fg7z\" (UniqueName: \"kubernetes.io/projected/fe6653ea-3449-4f95-9ebb-33c4d7ff658e-kube-api-access-8fg7z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qq9p\" (UID: \"fe6653ea-3449-4f95-9ebb-33c4d7ff658e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:36.897889 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:36.897853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:37.034207 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:37.034177    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"]
Jul 09 04:11:37.036129 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:11:37.036101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6653ea_3449_4f95_9ebb_33c4d7ff658e.slice/crio-25829f16f4208f2a6996e7bc481f930113f1055a9f5def619a042d7a864d5d35 WatchSource:0}: Error finding container 25829f16f4208f2a6996e7bc481f930113f1055a9f5def619a042d7a864d5d35: Status 404 returned error can't find the container with id 25829f16f4208f2a6996e7bc481f930113f1055a9f5def619a042d7a864d5d35
Jul 09 04:11:37.520782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:37.520703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p" event={"ID":"fe6653ea-3449-4f95-9ebb-33c4d7ff658e","Type":"ContainerStarted","Data":"25829f16f4208f2a6996e7bc481f930113f1055a9f5def619a042d7a864d5d35"}
Jul 09 04:11:37.552958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:37.552914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:38.534047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:38.533952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p" event={"ID":"fe6653ea-3449-4f95-9ebb-33c4d7ff658e","Type":"ContainerStarted","Data":"c6bdbfa95864d745486199f3e2bf93ce00f5602b3d5467c37fe64841b3dd7b1b"}
Jul 09 04:11:38.552640 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:38.552588    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p" podStartSLOduration=1.413825674 podStartE2EDuration="2.552573902s" totalImagesPullingTime="1.138748228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:38.549913892 +0000 UTC m=+697.824291800" watchObservedRunningTime="2026-07-09 04:11:38.552573902 +0000 UTC m=+697.826951810"
Jul 09 04:11:46.898032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:46.897994    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:46.898710 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:46.898115    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:46.910657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:46.910633    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:47.593297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:47.593267    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qq9p"
Jul 09 04:11:48.789193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:48.789162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:54.712543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:11:54.712440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:07.513167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:12:07.513134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:14.412809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:12:14.412775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:15.529284 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:15.529249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:43.905431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:43.905399    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f8869bc4d-jt4kx"]
Jul 09 04:13:43.909273 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:43.909254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:43.916963 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:43.916942    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f8869bc4d-jt4kx"]
Jul 09 04:13:44.033962 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.033928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-tls-cert\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.034106 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.034065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdrcr\" (UniqueName: \"kubernetes.io/projected/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-kube-api-access-vdrcr\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.135622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.135581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdrcr\" (UniqueName: \"kubernetes.io/projected/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-kube-api-access-vdrcr\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.135784 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.135678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-tls-cert\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.138180 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.138155    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-tls-cert\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.142299 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.142277    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdrcr\" (UniqueName: \"kubernetes.io/projected/38e18490-3acd-4e52-b2b6-ca7ac1b3faeb-kube-api-access-vdrcr\") pod \"authorino-f8869bc4d-jt4kx\" (UID: \"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb\") " pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.219413 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.219349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f8869bc4d-jt4kx"
Jul 09 04:13:44.347134 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:44.347111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f8869bc4d-jt4kx"]
Jul 09 04:13:44.350008 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:13:44.349974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38e18490_3acd_4e52_b2b6_ca7ac1b3faeb.slice/crio-c30f168c4ce10f115433023920958ce9b9ed10a11086b899f9f9ffb96209678d WatchSource:0}: Error finding container c30f168c4ce10f115433023920958ce9b9ed10a11086b899f9f9ffb96209678d: Status 404 returned error can't find the container with id c30f168c4ce10f115433023920958ce9b9ed10a11086b899f9f9ffb96209678d
Jul 09 04:13:45.246152 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.246110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f8869bc4d-jt4kx" event={"ID":"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb","Type":"ContainerStarted","Data":"f47d64276c352a9ba22b07002163e928250abd8ce5a320eadbe2a6af8a08f402"}
Jul 09 04:13:45.246641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.246163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f8869bc4d-jt4kx" event={"ID":"38e18490-3acd-4e52-b2b6-ca7ac1b3faeb","Type":"ContainerStarted","Data":"c30f168c4ce10f115433023920958ce9b9ed10a11086b899f9f9ffb96209678d"}
Jul 09 04:13:45.262646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.262581    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f8869bc4d-jt4kx" podStartSLOduration=1.870140785 podStartE2EDuration="2.26255957s" totalImagesPullingTime="392.418785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:45.258876841 +0000 UTC m=+824.533254774" watchObservedRunningTime="2026-07-09 04:13:45.26255957 +0000 UTC m=+824.536937479"
Jul 09 04:13:45.321196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.321156    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:13:45.321681 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.321628    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-54b988b6cc-x47pq" podUID="cbe53063-fbfe-40e4-9489-0095c64495d1" containerName="authorino" containerID="cri-o://b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9" gracePeriod=30
Jul 09 04:13:45.604569 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.604541    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:13:45.750709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.750614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert\") pod \"cbe53063-fbfe-40e4-9489-0095c64495d1\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") "
Jul 09 04:13:45.750709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.750676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kqjx\" (UniqueName: \"kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx\") pod \"cbe53063-fbfe-40e4-9489-0095c64495d1\" (UID: \"cbe53063-fbfe-40e4-9489-0095c64495d1\") "
Jul 09 04:13:45.753170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.753140    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx" (OuterVolumeSpecName: "kube-api-access-8kqjx") pod "cbe53063-fbfe-40e4-9489-0095c64495d1" (UID: "cbe53063-fbfe-40e4-9489-0095c64495d1"). InnerVolumeSpecName "kube-api-access-8kqjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:13:45.762289 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.762265    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cbe53063-fbfe-40e4-9489-0095c64495d1" (UID: "cbe53063-fbfe-40e4-9489-0095c64495d1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:13:45.794038 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.793997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:45.852399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.852359    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbe53063-fbfe-40e4-9489-0095c64495d1-tls-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:13:45.852399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.852400    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kqjx\" (UniqueName: \"kubernetes.io/projected/cbe53063-fbfe-40e4-9489-0095c64495d1-kube-api-access-8kqjx\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:13:45.886975 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:45.886935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:46.254205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.254164    2577 generic.go:356] "Generic (PLEG): container finished" podID="cbe53063-fbfe-40e4-9489-0095c64495d1" containerID="b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9" exitCode=0
Jul 09 04:13:46.254815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.254213    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b988b6cc-x47pq"
Jul 09 04:13:46.254815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.254215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b988b6cc-x47pq" event={"ID":"cbe53063-fbfe-40e4-9489-0095c64495d1","Type":"ContainerDied","Data":"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"}
Jul 09 04:13:46.254815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.254259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b988b6cc-x47pq" event={"ID":"cbe53063-fbfe-40e4-9489-0095c64495d1","Type":"ContainerDied","Data":"9f744d9981172c31277d7c9567942642310c1a278b145efcf676ff4ad68e2dcb"}
Jul 09 04:13:46.254815 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.254276    2577 scope.go:117] "RemoveContainer" containerID="b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"
Jul 09 04:13:46.264881 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.264864    2577 scope.go:117] "RemoveContainer" containerID="b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"
Jul 09 04:13:46.265153 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:13:46.265136    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9\": container with ID starting with b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9 not found: ID does not exist" containerID="b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"
Jul 09 04:13:46.265204 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.265162    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9"} err="failed to get container status \"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9\": rpc error: code = NotFound desc = could not find container \"b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9\": container with ID starting with b86971f20262b310ecc0316b9b32089925c54bcde3fc0b3e0c37c90cde0251e9 not found: ID does not exist"
Jul 09 04:13:46.276366 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.276344    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:13:46.279984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.279954    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54b988b6cc-x47pq"]
Jul 09 04:13:46.967779 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:46.967737    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:47.057633 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:47.057599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:47.181296 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:47.181257    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbe53063-fbfe-40e4-9489-0095c64495d1" path="/var/lib/kubelet/pods/cbe53063-fbfe-40e4-9489-0095c64495d1/volumes"
Jul 09 04:13:56.680490 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:56.680446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:56.732441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:13:56.732405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.602291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.602215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.701732 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.701685    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:14:56.701960 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.701908    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-68d56bd64d-czdbs" podUID="6cda9ee4-71b2-468a-841d-bb9416042098" containerName="manager" containerID="cri-o://a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e" gracePeriod=30
Jul 09 04:14:56.959727 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.959703    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:14:56.985393 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.985369    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d598t\" (UniqueName: \"kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t\") pod \"6cda9ee4-71b2-468a-841d-bb9416042098\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") "
Jul 09 04:14:56.985555 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.985457    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert\") pod \"6cda9ee4-71b2-468a-841d-bb9416042098\" (UID: \"6cda9ee4-71b2-468a-841d-bb9416042098\") "
Jul 09 04:14:56.987447 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.987415    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t" (OuterVolumeSpecName: "kube-api-access-d598t") pod "6cda9ee4-71b2-468a-841d-bb9416042098" (UID: "6cda9ee4-71b2-468a-841d-bb9416042098"). InnerVolumeSpecName "kube-api-access-d598t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:14:56.987557 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:56.987422    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6cda9ee4-71b2-468a-841d-bb9416042098" (UID: "6cda9ee4-71b2-468a-841d-bb9416042098"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:14:57.087061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.087029    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cda9ee4-71b2-468a-841d-bb9416042098-webhook-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:14:57.087061 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.087056    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d598t\" (UniqueName: \"kubernetes.io/projected/6cda9ee4-71b2-468a-841d-bb9416042098-kube-api-access-d598t\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:14:57.656491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.656441    2577 generic.go:356] "Generic (PLEG): container finished" podID="6cda9ee4-71b2-468a-841d-bb9416042098" containerID="a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e" exitCode=0
Jul 09 04:14:57.656944 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.656506    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-czdbs"
Jul 09 04:14:57.656944 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.656538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-czdbs" event={"ID":"6cda9ee4-71b2-468a-841d-bb9416042098","Type":"ContainerDied","Data":"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"}
Jul 09 04:14:57.656944 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.656602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-czdbs" event={"ID":"6cda9ee4-71b2-468a-841d-bb9416042098","Type":"ContainerDied","Data":"833a9cfaac8f8c365ff84c3366cbfee5c17c1ededcabb8c3064c36ffba585684"}
Jul 09 04:14:57.656944 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.656637    2577 scope.go:117] "RemoveContainer" containerID="a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"
Jul 09 04:14:57.665910 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.665894    2577 scope.go:117] "RemoveContainer" containerID="a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"
Jul 09 04:14:57.666173 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:14:57.666157    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e\": container with ID starting with a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e not found: ID does not exist" containerID="a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"
Jul 09 04:14:57.666219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.666184    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e"} err="failed to get container status \"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e\": rpc error: code = NotFound desc = could not find container \"a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e\": container with ID starting with a5d07ea9e7862a7213a7a7a8c09de276a9432cac3292f887322f1298a24e0a5e not found: ID does not exist"
Jul 09 04:14:57.683129 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.683103    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:14:57.686939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:57.686907    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-czdbs"]
Jul 09 04:14:58.320258 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.320224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:58.433373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.433343    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68d56bd64d-szkw4"]
Jul 09 04:14:58.433907 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.433879    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbe53063-fbfe-40e4-9489-0095c64495d1" containerName="authorino"
Jul 09 04:14:58.433907 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.433905    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbe53063-fbfe-40e4-9489-0095c64495d1" containerName="authorino"
Jul 09 04:14:58.434124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.433921    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cda9ee4-71b2-468a-841d-bb9416042098" containerName="manager"
Jul 09 04:14:58.434124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.433931    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cda9ee4-71b2-468a-841d-bb9416042098" containerName="manager"
Jul 09 04:14:58.434124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.434025    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cda9ee4-71b2-468a-841d-bb9416042098" containerName="manager"
Jul 09 04:14:58.434124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.434035    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbe53063-fbfe-40e4-9489-0095c64495d1" containerName="authorino"
Jul 09 04:14:58.438440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.438423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.440107 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.440082    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-thkpz\""
Jul 09 04:14:58.440219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.440121    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 04:14:58.440552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.440536    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 04:14:58.450410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.447945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-szkw4"]
Jul 09 04:14:58.501321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.501295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llkhc\" (UniqueName: \"kubernetes.io/projected/63cc533b-e9b6-46d6-b461-6ef612bce631-kube-api-access-llkhc\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.501481 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.501384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63cc533b-e9b6-46d6-b461-6ef612bce631-webhook-cert\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.602228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.602203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llkhc\" (UniqueName: \"kubernetes.io/projected/63cc533b-e9b6-46d6-b461-6ef612bce631-kube-api-access-llkhc\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.602347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.602262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63cc533b-e9b6-46d6-b461-6ef612bce631-webhook-cert\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.604631 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.604609    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63cc533b-e9b6-46d6-b461-6ef612bce631-webhook-cert\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.609403 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.609386    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llkhc\" (UniqueName: \"kubernetes.io/projected/63cc533b-e9b6-46d6-b461-6ef612bce631-kube-api-access-llkhc\") pod \"maas-controller-68d56bd64d-szkw4\" (UID: \"63cc533b-e9b6-46d6-b461-6ef612bce631\") " pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.751591 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.751557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:58.884919 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:58.884892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d56bd64d-szkw4"]
Jul 09 04:14:58.887428 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:14:58.887402    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cc533b_e9b6_46d6_b461_6ef612bce631.slice/crio-f20f4ae4b97d90da609bca0335055513b7f36900f00d44ab0f876b0ae717dde2 WatchSource:0}: Error finding container f20f4ae4b97d90da609bca0335055513b7f36900f00d44ab0f876b0ae717dde2: Status 404 returned error can't find the container with id f20f4ae4b97d90da609bca0335055513b7f36900f00d44ab0f876b0ae717dde2
Jul 09 04:14:59.179781 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:59.179702    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cda9ee4-71b2-468a-841d-bb9416042098" path="/var/lib/kubelet/pods/6cda9ee4-71b2-468a-841d-bb9416042098/volumes"
Jul 09 04:14:59.669532 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:59.669497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-szkw4" event={"ID":"63cc533b-e9b6-46d6-b461-6ef612bce631","Type":"ContainerStarted","Data":"62247140f9add5651da8c23794f4a03cac14b6fc89c9b14e30a953174aaaa9ea"}
Jul 09 04:14:59.669697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:59.669548    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d56bd64d-szkw4" event={"ID":"63cc533b-e9b6-46d6-b461-6ef612bce631","Type":"ContainerStarted","Data":"f20f4ae4b97d90da609bca0335055513b7f36900f00d44ab0f876b0ae717dde2"}
Jul 09 04:14:59.669697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:59.669597    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:14:59.685980 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:14:59.685885    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68d56bd64d-szkw4" podStartSLOduration=1.15380071 podStartE2EDuration="1.68586974s" totalImagesPullingTime="532.06903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:14:59.684007165 +0000 UTC m=+898.958385103" watchObservedRunningTime="2026-07-09 04:14:59.68586974 +0000 UTC m=+898.960247647"
Jul 09 04:15:00.141685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.141654    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:15:00.145356 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.145337    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:15:00.152162 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.152138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:15:00.216592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.216561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99267\" (UniqueName: \"kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267\") pod \"maas-api-key-cleanup-29726175-fzx58\" (UID: \"eeb32d62-404d-49f3-a028-79b943370397\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:15:00.317320 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.317288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99267\" (UniqueName: \"kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267\") pod \"maas-api-key-cleanup-29726175-fzx58\" (UID: \"eeb32d62-404d-49f3-a028-79b943370397\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:15:00.324834 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.324813    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99267\" (UniqueName: \"kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267\") pod \"maas-api-key-cleanup-29726175-fzx58\" (UID: \"eeb32d62-404d-49f3-a028-79b943370397\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:15:00.456989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.456913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:15:00.585563 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:15:00.585525    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb32d62_404d_49f3_a028_79b943370397.slice/crio-d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9 WatchSource:0}: Error finding container d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9: Status 404 returned error can't find the container with id d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9
Jul 09 04:15:00.586065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.586041    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:15:00.675882 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:00.675854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" event={"ID":"eeb32d62-404d-49f3-a028-79b943370397","Type":"ContainerStarted","Data":"d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9"}
Jul 09 04:15:01.118920 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:01.118896    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:15:01.120967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:01.120946    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:15:03.695730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:03.695688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" event={"ID":"eeb32d62-404d-49f3-a028-79b943370397","Type":"ContainerStarted","Data":"f529eb575a7d8f7efc4ee563662724264dfe305eb03b87ec59f66aa4aa6a84b5"}
Jul 09 04:15:03.711112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:03.711063    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" podStartSLOduration=1.019488131 podStartE2EDuration="3.711043271s" totalImagesPullingTime="2.69155514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:03.70995867 +0000 UTC m=+902.984336578" watchObservedRunningTime="2026-07-09 04:15:03.711043271 +0000 UTC m=+902.985421180"
Jul 09 04:15:10.680640 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:10.680610    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68d56bd64d-szkw4"
Jul 09 04:15:26.450189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:26.450156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:28.824896 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:28.824836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:28.928155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:28.928118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:36.765688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:36.765638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:36.866802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:36.866760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:36.968064 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:36.968023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:42.742005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:42.741971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:42.808781 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:42.808750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:42.952856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:42.952828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:43.042273 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:43.042153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:43.143607 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:43.143571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:48.650565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:48.650531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:48.714035 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:15:48.714007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:00.013089 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:00.013058    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:16:00.013537 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:00.013308    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" podUID="eeb32d62-404d-49f3-a028-79b943370397" containerName="cleanup" containerID="cri-o://f529eb575a7d8f7efc4ee563662724264dfe305eb03b87ec59f66aa4aa6a84b5" gracePeriod=30
Jul 09 04:16:03.191571 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:03.191524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:04.416733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:04.416679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:30.191278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.191246    2577 generic.go:356] "Generic (PLEG): container finished" podID="eeb32d62-404d-49f3-a028-79b943370397" containerID="f529eb575a7d8f7efc4ee563662724264dfe305eb03b87ec59f66aa4aa6a84b5" exitCode=137
Jul 09 04:16:30.191724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.191317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" event={"ID":"eeb32d62-404d-49f3-a028-79b943370397","Type":"ContainerDied","Data":"f529eb575a7d8f7efc4ee563662724264dfe305eb03b87ec59f66aa4aa6a84b5"}
Jul 09 04:16:30.191724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.191379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58" event={"ID":"eeb32d62-404d-49f3-a028-79b943370397","Type":"ContainerDied","Data":"d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9"}
Jul 09 04:16:30.191724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.191402    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d254c2ce407fff4eb036ad03c933eaaf865f05769d321c380c3deb7eefed12a9"
Jul 09 04:16:30.206233 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.206215    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:16:30.321170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.321084    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99267\" (UniqueName: \"kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267\") pod \"eeb32d62-404d-49f3-a028-79b943370397\" (UID: \"eeb32d62-404d-49f3-a028-79b943370397\") "
Jul 09 04:16:30.323251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.323228    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267" (OuterVolumeSpecName: "kube-api-access-99267") pod "eeb32d62-404d-49f3-a028-79b943370397" (UID: "eeb32d62-404d-49f3-a028-79b943370397"). InnerVolumeSpecName "kube-api-access-99267". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:16:30.422550 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:30.422514    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99267\" (UniqueName: \"kubernetes.io/projected/eeb32d62-404d-49f3-a028-79b943370397-kube-api-access-99267\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:16:31.196796 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:31.196772    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"
Jul 09 04:16:31.216132 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:31.216102    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:16:31.221697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:31.221671    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-fzx58"]
Jul 09 04:16:33.180091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:16:33.180057    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeb32d62-404d-49f3-a028-79b943370397" path="/var/lib/kubelet/pods/eeb32d62-404d-49f3-a028-79b943370397/volumes"
Jul 09 04:18:21.559399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:18:21.559355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:04.121041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:04.120980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:04.221807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:04.221768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:37.947893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:37.947682    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:19:37.948744 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:37.948572    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" podUID="d702dad3-a57e-4a83-b891-626edfff2580" containerName="manager" containerID="cri-o://389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b" gracePeriod=10
Jul 09 04:19:40.711682 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.711655    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:19:40.737130 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.737096    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhdgt\" (UniqueName: \"kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt\") pod \"d702dad3-a57e-4a83-b891-626edfff2580\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") "
Jul 09 04:19:40.737275 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.737152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume\") pod \"d702dad3-a57e-4a83-b891-626edfff2580\" (UID: \"d702dad3-a57e-4a83-b891-626edfff2580\") "
Jul 09 04:19:40.737615 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.737578    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d702dad3-a57e-4a83-b891-626edfff2580" (UID: "d702dad3-a57e-4a83-b891-626edfff2580"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:40.739248 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.739216    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt" (OuterVolumeSpecName: "kube-api-access-xhdgt") pod "d702dad3-a57e-4a83-b891-626edfff2580" (UID: "d702dad3-a57e-4a83-b891-626edfff2580"). InnerVolumeSpecName "kube-api-access-xhdgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:19:40.837910 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.837842    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhdgt\" (UniqueName: \"kubernetes.io/projected/d702dad3-a57e-4a83-b891-626edfff2580-kube-api-access-xhdgt\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:19:40.837910 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:40.837867    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d702dad3-a57e-4a83-b891-626edfff2580-extensions-socket-volume\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:19:41.295662 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.295626    2577 generic.go:356] "Generic (PLEG): container finished" podID="d702dad3-a57e-4a83-b891-626edfff2580" containerID="389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b" exitCode=0
Jul 09 04:19:41.295838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.295679    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"
Jul 09 04:19:41.295838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.295719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" event={"ID":"d702dad3-a57e-4a83-b891-626edfff2580","Type":"ContainerDied","Data":"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"}
Jul 09 04:19:41.295838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.295774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4" event={"ID":"d702dad3-a57e-4a83-b891-626edfff2580","Type":"ContainerDied","Data":"01420c6b83e69d09d5b807f03b28b3674a374b64f16820970b482823d72f0791"}
Jul 09 04:19:41.295838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.295796    2577 scope.go:117] "RemoveContainer" containerID="389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"
Jul 09 04:19:41.305478 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.305456    2577 scope.go:117] "RemoveContainer" containerID="389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"
Jul 09 04:19:41.305753 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:19:41.305732    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b\": container with ID starting with 389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b not found: ID does not exist" containerID="389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"
Jul 09 04:19:41.305811 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.305758    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b"} err="failed to get container status \"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b\": rpc error: code = NotFound desc = could not find container \"389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b\": container with ID starting with 389a0ca6529be261089946c1092207e80dba1ac9e962a4664a20c8e8b9dbf41b not found: ID does not exist"
Jul 09 04:19:41.311406 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.311384    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:19:41.314518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:41.314458    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jznk4"]
Jul 09 04:19:43.186487 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:19:43.186448    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d702dad3-a57e-4a83-b891-626edfff2580" path="/var/lib/kubelet/pods/d702dad3-a57e-4a83-b891-626edfff2580/volumes"
Jul 09 04:20:01.157237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:01.157203    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:20:01.161357 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:01.161333    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:20:39.052671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.052635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"]
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053088    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d702dad3-a57e-4a83-b891-626edfff2580" containerName="manager"
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053100    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d702dad3-a57e-4a83-b891-626edfff2580" containerName="manager"
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053111    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeb32d62-404d-49f3-a028-79b943370397" containerName="cleanup"
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053116    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeb32d62-404d-49f3-a028-79b943370397" containerName="cleanup"
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053177    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeb32d62-404d-49f3-a028-79b943370397" containerName="cleanup"
Jul 09 04:20:39.054935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.053188    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d702dad3-a57e-4a83-b891-626edfff2580" containerName="manager"
Jul 09 04:20:39.056244 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.056229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.058394 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.058375    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kmxl5\""
Jul 09 04:20:39.066890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.066865    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"]
Jul 09 04:20:39.142704 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.142664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.142888 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.142722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkk6p\" (UniqueName: \"kubernetes.io/projected/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-kube-api-access-bkk6p\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.243810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.243767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.243810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.243814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkk6p\" (UniqueName: \"kubernetes.io/projected/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-kube-api-access-bkk6p\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.244154 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.244132    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.252072 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.252050    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkk6p\" (UniqueName: \"kubernetes.io/projected/b38f252c-3fcc-426d-8ea2-79f6fe3d378b-kube-api-access-bkk6p\") pod \"kuadrant-operator-controller-manager-86878c769f-dbmn2\" (UID: \"b38f252c-3fcc-426d-8ea2-79f6fe3d378b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.368483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.368441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.512414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.512381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"]
Jul 09 04:20:39.516236 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:20:39.516209    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38f252c_3fcc_426d_8ea2_79f6fe3d378b.slice/crio-262d9f2c87f9d86855b7af1ad16b8bdd84f17bbc7b210e9fd304c69049d948ff WatchSource:0}: Error finding container 262d9f2c87f9d86855b7af1ad16b8bdd84f17bbc7b210e9fd304c69049d948ff: Status 404 returned error can't find the container with id 262d9f2c87f9d86855b7af1ad16b8bdd84f17bbc7b210e9fd304c69049d948ff
Jul 09 04:20:39.623035 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.622946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2" event={"ID":"b38f252c-3fcc-426d-8ea2-79f6fe3d378b","Type":"ContainerStarted","Data":"ebe1b9ce9a9f80bb72b96074c7e6cd0c2b7aed3c90f904cbaaa8ad9e6be86847"}
Jul 09 04:20:39.623035 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.623011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:20:39.623214 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.623036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2" event={"ID":"b38f252c-3fcc-426d-8ea2-79f6fe3d378b","Type":"ContainerStarted","Data":"262d9f2c87f9d86855b7af1ad16b8bdd84f17bbc7b210e9fd304c69049d948ff"}
Jul 09 04:20:39.640686 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:39.640626    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2" podStartSLOduration=0.640602611 podStartE2EDuration="640.602611ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:20:39.639337965 +0000 UTC m=+1238.913715872" watchObservedRunningTime="2026-07-09 04:20:39.640602611 +0000 UTC m=+1238.914980520"
Jul 09 04:20:50.631376 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:20:50.631342    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbmn2"
Jul 09 04:21:40.953561 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:21:40.953530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:21:42.634400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:21:42.634366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:22:01.266069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:22:01.266036    2577 scope.go:117] "RemoveContainer" containerID="f529eb575a7d8f7efc4ee563662724264dfe305eb03b87ec59f66aa4aa6a84b5"
Jul 09 04:24:24.501688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:24:24.501660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:24:42.030350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:24:42.030317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:25:01.200856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:25:01.200828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:25:01.205598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:25:01.205574    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:25:53.953506 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:25:53.953447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:23.052297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:23.052262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:30.406032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:30.405991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:38.663139 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:38.663102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:43.710789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:43.710752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:46.388113 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:46.388083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:46.495332 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:46.495300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:46.898333 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:46.898282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:52.579737 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.579704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:52.682036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.681999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:52.780885 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.780858    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:26:52.785676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.785657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.788821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.788800    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-05e0d0ab-openshift-default-dockercfg-8t88v\""
Jul 09 04:26:52.812182 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.812121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:26:52.820197 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.820178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:52.822461 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtsx\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822847 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822847 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822847 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.822847 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.822700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.918461 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.918435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:52.923625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtsx\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.923958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.923966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.924084    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.924175    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924305 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.924284    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.924389    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.924575 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.924431    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.925864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.925840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.926161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.926144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.933198 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.933174    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:52.933389 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:52.933365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtsx\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx\") pod \"e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") " pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:53.096505 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.096452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:53.236537 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.236515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:26:53.241520 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:26:53.239400    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b96b92a_665e_4c0e_b1e0_924fd46402de.slice/crio-a8cc30d215be20e760e7fbbc4f86d95c7d34e7861f845b8f9fee423920e0c396 WatchSource:0}: Error finding container a8cc30d215be20e760e7fbbc4f86d95c7d34e7861f845b8f9fee423920e0c396: Status 404 returned error can't find the container with id a8cc30d215be20e760e7fbbc4f86d95c7d34e7861f845b8f9fee423920e0c396
Jul 09 04:26:53.248000 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.247959    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:26:53.248091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.248046    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:26:53.248143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.248125    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:26:53.748926 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.748892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" event={"ID":"7b96b92a-665e-4c0e-b1e0-924fd46402de","Type":"ContainerStarted","Data":"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"}
Jul 09 04:26:53.749392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.748969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" event={"ID":"7b96b92a-665e-4c0e-b1e0-924fd46402de","Type":"ContainerStarted","Data":"a8cc30d215be20e760e7fbbc4f86d95c7d34e7861f845b8f9fee423920e0c396"}
Jul 09 04:26:53.771089 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.771042    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" podStartSLOduration=1.7710299969999999 podStartE2EDuration="1.771029997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:26:53.76851394 +0000 UTC m=+1613.042891876" watchObservedRunningTime="2026-07-09 04:26:53.771029997 +0000 UTC m=+1613.045407905"
Jul 09 04:26:53.974059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:53.974025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.049718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.049628    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:26:54.053659 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.053637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.055558 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.055532    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-05e0d0ab-serving-cert\""
Jul 09 04:26:54.064939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.064918    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:26:54.077917 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.077888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.096787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.096768    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:54.101663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.101643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:54.134033 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.134004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27jmc\" (UniqueName: \"kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.134033 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.134035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.134258 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.134067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.144714 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.144674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.235155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.235118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27jmc\" (UniqueName: \"kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.235455 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.235169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.235455 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.235203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.235455 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:26:54.235305    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-serving-cert: secret "maas-api-e2e-ait-05e0d0ab-serving-cert" not found
Jul 09 04:26:54.235455 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:26:54.235379    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls podName:40394fef-ecfd-47ec-bfd9-03295908cfb3 nodeName:}" failed. No retries permitted until 2026-07-09 04:26:54.735355502 +0000 UTC m=+1614.009733394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls") pod "maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" (UID: "40394fef-ecfd-47ec-bfd9-03295908cfb3") : secret "maas-api-e2e-ait-05e0d0ab-serving-cert" not found
Jul 09 04:26:54.236010 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.235993    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.257048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.257021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27jmc\" (UniqueName: \"kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.405264 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.405236    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.592655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.592619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.692319 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.692242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:54.739742 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.739711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.742359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.742335    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") pod \"maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:54.754951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.754930    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:54.755888 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.755873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:26:54.966646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:54.966557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:55.106162 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.106137    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:26:55.107966 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:26:55.107936    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40394fef_ecfd_47ec_bfd9_03295908cfb3.slice/crio-da17ff69c654d968ccb52ef205326acc91bcef6f8f64166f5058fe9b26c7a342 WatchSource:0}: Error finding container da17ff69c654d968ccb52ef205326acc91bcef6f8f64166f5058fe9b26c7a342: Status 404 returned error can't find the container with id da17ff69c654d968ccb52ef205326acc91bcef6f8f64166f5058fe9b26c7a342
Jul 09 04:26:55.109393 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.109376    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:26:55.762635 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.762597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" event={"ID":"40394fef-ecfd-47ec-bfd9-03295908cfb3","Type":"ContainerStarted","Data":"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"}
Jul 09 04:26:55.762635 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.762652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" event={"ID":"40394fef-ecfd-47ec-bfd9-03295908cfb3","Type":"ContainerStarted","Data":"da17ff69c654d968ccb52ef205326acc91bcef6f8f64166f5058fe9b26c7a342"}
Jul 09 04:26:55.763170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.762679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:55.780186 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:55.780145    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" podStartSLOduration=1.3160613460000001 podStartE2EDuration="1.780130083s" totalImagesPullingTime="464.068737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:26:55.777269858 +0000 UTC m=+1615.051647785" watchObservedRunningTime="2026-07-09 04:26:55.780130083 +0000 UTC m=+1615.054507990"
Jul 09 04:26:56.366423 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:56.366392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:58.960595 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:58.960558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:59.121964 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.121928    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:26:59.122138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.122097    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" podUID="7b96b92a-665e-4c0e-b1e0-924fd46402de" containerName="istio-proxy" containerID="cri-o://c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176" gracePeriod=30
Jul 09 04:26:59.148363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.148321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:59.235157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.235064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:59.336416 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.336383    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:26:59.336863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.336836    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" podUID="40394fef-ecfd-47ec-bfd9-03295908cfb3" containerName="maas-api" containerID="cri-o://7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981" gracePeriod=30
Jul 09 04:26:59.351496 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.350949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:26:59.615861 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.615840    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:59.682494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.682442    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27jmc\" (UniqueName: \"kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc\") pod \"40394fef-ecfd-47ec-bfd9-03295908cfb3\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") "
Jul 09 04:26:59.682651 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.682581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca\") pod \"40394fef-ecfd-47ec-bfd9-03295908cfb3\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") "
Jul 09 04:26:59.682651 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.682608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") pod \"40394fef-ecfd-47ec-bfd9-03295908cfb3\" (UID: \"40394fef-ecfd-47ec-bfd9-03295908cfb3\") "
Jul 09 04:26:59.682982 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.682943    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40394fef-ecfd-47ec-bfd9-03295908cfb3" (UID: "40394fef-ecfd-47ec-bfd9-03295908cfb3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:26:59.684435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.684415    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc" (OuterVolumeSpecName: "kube-api-access-27jmc") pod "40394fef-ecfd-47ec-bfd9-03295908cfb3" (UID: "40394fef-ecfd-47ec-bfd9-03295908cfb3"). InnerVolumeSpecName "kube-api-access-27jmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:26:59.684670 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.684651    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40394fef-ecfd-47ec-bfd9-03295908cfb3" (UID: "40394fef-ecfd-47ec-bfd9-03295908cfb3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:26:59.783563 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.783460    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40394fef-ecfd-47ec-bfd9-03295908cfb3-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:26:59.783563 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.783558    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40394fef-ecfd-47ec-bfd9-03295908cfb3-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:26:59.783754 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.783574    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27jmc\" (UniqueName: \"kubernetes.io/projected/40394fef-ecfd-47ec-bfd9-03295908cfb3-kube-api-access-27jmc\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:26:59.785829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.785805    2577 generic.go:356] "Generic (PLEG): container finished" podID="40394fef-ecfd-47ec-bfd9-03295908cfb3" containerID="7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981" exitCode=0
Jul 09 04:26:59.785939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.785853    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"
Jul 09 04:26:59.785939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.785876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" event={"ID":"40394fef-ecfd-47ec-bfd9-03295908cfb3","Type":"ContainerDied","Data":"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"}
Jul 09 04:26:59.785939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.785923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt" event={"ID":"40394fef-ecfd-47ec-bfd9-03295908cfb3","Type":"ContainerDied","Data":"da17ff69c654d968ccb52ef205326acc91bcef6f8f64166f5058fe9b26c7a342"}
Jul 09 04:26:59.786040 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.785946    2577 scope.go:117] "RemoveContainer" containerID="7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"
Jul 09 04:26:59.796435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.796413    2577 scope.go:117] "RemoveContainer" containerID="7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"
Jul 09 04:26:59.796724 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:26:59.796706    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981\": container with ID starting with 7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981 not found: ID does not exist" containerID="7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"
Jul 09 04:26:59.796868 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.796732    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981"} err="failed to get container status \"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981\": rpc error: code = NotFound desc = could not find container \"7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981\": container with ID starting with 7ddb81d072552dbcd903b4c389d07244fcd928ebf144cf3d7d7d1689a81cc981 not found: ID does not exist"
Jul 09 04:26:59.806752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.806730    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:26:59.810325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:26:59.810306    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05e0d0ab-5fb6f97b9d-r8gpt"]
Jul 09 04:27:00.478874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:00.478845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:01.181137 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:01.181105    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40394fef-ecfd-47ec-bfd9-03295908cfb3" path="/var/lib/kubelet/pods/40394fef-ecfd-47ec-bfd9-03295908cfb3/volumes"
Jul 09 04:27:04.984669 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:04.984634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:05.051353 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:05.051321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:05.214129 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:05.214093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:05.309703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:05.309620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:05.974306 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:05.974255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:27:06.042056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042029    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042181    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042457    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:06.042514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042504    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:06.042992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042508    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:27:06.042992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042522    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:06.042992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.042992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.042654    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmtsx\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx\") pod \"7b96b92a-665e-4c0e-b1e0-924fd46402de\" (UID: \"7b96b92a-665e-4c0e-b1e0-924fd46402de\") "
Jul 09 04:27:06.043149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.043003    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.043149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.043010    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:06.043149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.043025    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.043149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.043039    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b96b92a-665e-4c0e-b1e0-924fd46402de-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.043149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.043053    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.044238 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.044213    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:27:06.044707 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.044679    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:06.044798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.044738    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx" (OuterVolumeSpecName: "kube-api-access-jmtsx") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "kube-api-access-jmtsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:06.044798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.044762    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b96b92a-665e-4c0e-b1e0-924fd46402de" (UID: "7b96b92a-665e-4c0e-b1e0-924fd46402de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:06.144441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.144406    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.144441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.144437    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.144441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.144447    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtsx\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-kube-api-access-jmtsx\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.144700 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.144457    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.144700 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.144490    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b96b92a-665e-4c0e-b1e0-924fd46402de-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:06.290647 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.290606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:06.389262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.389232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:06.486059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.485982    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:06.486507 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486489    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b96b92a-665e-4c0e-b1e0-924fd46402de" containerName="istio-proxy"
Jul 09 04:27:06.486568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486508    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b96b92a-665e-4c0e-b1e0-924fd46402de" containerName="istio-proxy"
Jul 09 04:27:06.486568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486531    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40394fef-ecfd-47ec-bfd9-03295908cfb3" containerName="maas-api"
Jul 09 04:27:06.486568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486536    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="40394fef-ecfd-47ec-bfd9-03295908cfb3" containerName="maas-api"
Jul 09 04:27:06.486665 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486630    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="40394fef-ecfd-47ec-bfd9-03295908cfb3" containerName="maas-api"
Jul 09 04:27:06.486665 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.486637    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b96b92a-665e-4c0e-b1e0-924fd46402de" containerName="istio-proxy"
Jul 09 04:27:06.489972 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.489958    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.491781 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.491755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d6efa5ba-serving-cert\""
Jul 09 04:27:06.500493 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.500447    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:06.548292 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.548265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.548424 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.548300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg4xm\" (UniqueName: \"kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.548424 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.548381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.649603 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.649574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.649755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.649611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg4xm\" (UniqueName: \"kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.649755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.649665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.649755 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:27:06.649711    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-serving-cert: secret "maas-api-e2e-ait-d6efa5ba-serving-cert" not found
Jul 09 04:27:06.649889 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:27:06.649785    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls podName:a6fa77e0-2160-4141-af57-1136bb94f7b1 nodeName:}" failed. No retries permitted until 2026-07-09 04:27:07.149761159 +0000 UTC m=+1626.424139046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls") pod "maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" (UID: "a6fa77e0-2160-4141-af57-1136bb94f7b1") : secret "maas-api-e2e-ait-d6efa5ba-serving-cert" not found
Jul 09 04:27:06.650233 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.650217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.658901 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.658881    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg4xm\" (UniqueName: \"kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:06.830600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.830520    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b96b92a-665e-4c0e-b1e0-924fd46402de" containerID="c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176" exitCode=0
Jul 09 04:27:06.830600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.830574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"
Jul 09 04:27:06.830802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.830606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" event={"ID":"7b96b92a-665e-4c0e-b1e0-924fd46402de","Type":"ContainerDied","Data":"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"}
Jul 09 04:27:06.830802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.830661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp" event={"ID":"7b96b92a-665e-4c0e-b1e0-924fd46402de","Type":"ContainerDied","Data":"a8cc30d215be20e760e7fbbc4f86d95c7d34e7861f845b8f9fee423920e0c396"}
Jul 09 04:27:06.830802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.830695    2577 scope.go:117] "RemoveContainer" containerID="c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"
Jul 09 04:27:06.843138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.842943    2577 scope.go:117] "RemoveContainer" containerID="c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"
Jul 09 04:27:06.843218 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:27:06.843188    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176\": container with ID starting with c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176 not found: ID does not exist" containerID="c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"
Jul 09 04:27:06.843275 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.843212    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176"} err="failed to get container status \"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176\": rpc error: code = NotFound desc = could not find container \"c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176\": container with ID starting with c298dc818b8c9ec4e0529b67883f1c8f03f29e0a3a1c4dcb4423a0a6535ff176 not found: ID does not exist"
Jul 09 04:27:06.856302 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.856279    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:27:06.857419 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.857395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:06.858938 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.858917    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-05e0d0ab-openshift-default-7b9d47887c-rj4xp"]
Jul 09 04:27:06.921079 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:06.921055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:07.058121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.058090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:07.154331 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.154307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:07.156655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.156640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") pod \"maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:07.159400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.159376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:07.180051 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.180027    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b96b92a-665e-4c0e-b1e0-924fd46402de" path="/var/lib/kubelet/pods/7b96b92a-665e-4c0e-b1e0-924fd46402de/volumes"
Jul 09 04:27:07.402959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.402924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:07.559524 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.552378    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:07.575342 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:27:07.575314    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6fa77e0_2160_4141_af57_1136bb94f7b1.slice/crio-04a0a98672b40fe42bea6cf9ffd6a81abadf30ae896e2960e89db2048faa3a0f WatchSource:0}: Error finding container 04a0a98672b40fe42bea6cf9ffd6a81abadf30ae896e2960e89db2048faa3a0f: Status 404 returned error can't find the container with id 04a0a98672b40fe42bea6cf9ffd6a81abadf30ae896e2960e89db2048faa3a0f
Jul 09 04:27:07.839171 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:07.839077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" event={"ID":"a6fa77e0-2160-4141-af57-1136bb94f7b1","Type":"ContainerStarted","Data":"04a0a98672b40fe42bea6cf9ffd6a81abadf30ae896e2960e89db2048faa3a0f"}
Jul 09 04:27:08.388396 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:08.388366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:08.847161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:08.847129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" event={"ID":"a6fa77e0-2160-4141-af57-1136bb94f7b1","Type":"ContainerStarted","Data":"bc4a132a8bc831309997106c3d01d292d3c9fcecb9444ea892ee0fdc6aba6784"}
Jul 09 04:27:08.847336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:08.847269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:08.864002 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:08.863951    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" podStartSLOduration=2.46323441 podStartE2EDuration="2.863937611s" totalImagesPullingTime="400.703201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:27:08.862799823 +0000 UTC m=+1628.137177751" watchObservedRunningTime="2026-07-09 04:27:08.863937611 +0000 UTC m=+1628.138315518"
Jul 09 04:27:11.372817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.372781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:11.743155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.743065    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:11.743326 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.743282    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" podUID="a6fa77e0-2160-4141-af57-1136bb94f7b1" containerName="maas-api" containerID="cri-o://bc4a132a8bc831309997106c3d01d292d3c9fcecb9444ea892ee0fdc6aba6784" gracePeriod=30
Jul 09 04:27:11.763381 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.763359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:11.866395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.866370    2577 generic.go:356] "Generic (PLEG): container finished" podID="a6fa77e0-2160-4141-af57-1136bb94f7b1" containerID="bc4a132a8bc831309997106c3d01d292d3c9fcecb9444ea892ee0fdc6aba6784" exitCode=0
Jul 09 04:27:11.866525 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.866445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" event={"ID":"a6fa77e0-2160-4141-af57-1136bb94f7b1","Type":"ContainerDied","Data":"bc4a132a8bc831309997106c3d01d292d3c9fcecb9444ea892ee0fdc6aba6784"}
Jul 09 04:27:11.993660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:11.993604    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:12.097171 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.097139    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") pod \"a6fa77e0-2160-4141-af57-1136bb94f7b1\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") "
Jul 09 04:27:12.097359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.097176    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg4xm\" (UniqueName: \"kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm\") pod \"a6fa77e0-2160-4141-af57-1136bb94f7b1\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") "
Jul 09 04:27:12.097359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.097226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca\") pod \"a6fa77e0-2160-4141-af57-1136bb94f7b1\" (UID: \"a6fa77e0-2160-4141-af57-1136bb94f7b1\") "
Jul 09 04:27:12.097653 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.097625    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6fa77e0-2160-4141-af57-1136bb94f7b1" (UID: "a6fa77e0-2160-4141-af57-1136bb94f7b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:27:12.099214 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.099196    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6fa77e0-2160-4141-af57-1136bb94f7b1" (UID: "a6fa77e0-2160-4141-af57-1136bb94f7b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:27:12.099214 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.099195    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm" (OuterVolumeSpecName: "kube-api-access-fg4xm") pod "a6fa77e0-2160-4141-af57-1136bb94f7b1" (UID: "a6fa77e0-2160-4141-af57-1136bb94f7b1"). InnerVolumeSpecName "kube-api-access-fg4xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:12.198514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.198454    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6fa77e0-2160-4141-af57-1136bb94f7b1-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:12.198514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.198506    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg4xm\" (UniqueName: \"kubernetes.io/projected/a6fa77e0-2160-4141-af57-1136bb94f7b1-kube-api-access-fg4xm\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:12.198514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.198517    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6fa77e0-2160-4141-af57-1136bb94f7b1-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:12.873764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.873731    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"
Jul 09 04:27:12.873764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.873753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl" event={"ID":"a6fa77e0-2160-4141-af57-1136bb94f7b1","Type":"ContainerDied","Data":"04a0a98672b40fe42bea6cf9ffd6a81abadf30ae896e2960e89db2048faa3a0f"}
Jul 09 04:27:12.874232 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.873814    2577 scope.go:117] "RemoveContainer" containerID="bc4a132a8bc831309997106c3d01d292d3c9fcecb9444ea892ee0fdc6aba6784"
Jul 09 04:27:12.893754 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.893726    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:12.899087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:12.899066    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6efa5ba-9d966899f-t5xxl"]
Jul 09 04:27:13.180325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:13.180247    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6fa77e0-2160-4141-af57-1136bb94f7b1" path="/var/lib/kubelet/pods/a6fa77e0-2160-4141-af57-1136bb94f7b1/volumes"
Jul 09 04:27:17.483278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:17.483248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:17.578658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:17.578606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:19.296660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:19.296633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:23.728697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.728655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:23.830976 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.830941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:23.911969 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.911938    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:23.912435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.912421    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6fa77e0-2160-4141-af57-1136bb94f7b1" containerName="maas-api"
Jul 09 04:27:23.912489 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.912437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6fa77e0-2160-4141-af57-1136bb94f7b1" containerName="maas-api"
Jul 09 04:27:23.912532 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.912520    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6fa77e0-2160-4141-af57-1136bb94f7b1" containerName="maas-api"
Jul 09 04:27:23.916959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.916943    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:23.919426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.919402    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f532f789-openshift-default-dockercfg-jq8n4\""
Jul 09 04:27:23.929619 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.928384    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:23.948646 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:23.948625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:24.006806 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.006806 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfn8j\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.006950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.006950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.006950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.006950 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.007087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006946    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.007087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.006981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.007087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.007006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.044818 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.044786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:24.108405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfn8j\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108993 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108851    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.108993 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.108917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.109228 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.109200    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.109345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.109207    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.109565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.109546    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.110799 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.110768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.110939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.110920    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.116538 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.116518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.116761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.116744    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfn8j\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j\") pod \"e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") " pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.231059 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.231030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:24.379144 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.379116    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:24.381047 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:27:24.381014    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93b6b65_cf2c_4cab_aa72_ed4432396f87.slice/crio-3a81b4d72b79d298e93c249e8f30596a43e96c14b4be8062797ca1dd5ff2b4d2 WatchSource:0}: Error finding container 3a81b4d72b79d298e93c249e8f30596a43e96c14b4be8062797ca1dd5ff2b4d2: Status 404 returned error can't find the container with id 3a81b4d72b79d298e93c249e8f30596a43e96c14b4be8062797ca1dd5ff2b4d2
Jul 09 04:27:24.383210 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.383173    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:27:24.383284 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.383259    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:27:24.383321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.383304    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:27:24.947665 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.947619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" event={"ID":"e93b6b65-cf2c-4cab-aa72-ed4432396f87","Type":"ContainerStarted","Data":"db1b3bd4977caca0b8811bd71b339acd7c2633b383fe79fa1387209995f806b8"}
Jul 09 04:27:24.948118 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.947680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" event={"ID":"e93b6b65-cf2c-4cab-aa72-ed4432396f87","Type":"ContainerStarted","Data":"3a81b4d72b79d298e93c249e8f30596a43e96c14b4be8062797ca1dd5ff2b4d2"}
Jul 09 04:27:24.965100 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:24.965029    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" podStartSLOduration=1.965008117 podStartE2EDuration="1.965008117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:27:24.964282278 +0000 UTC m=+1644.238660175" watchObservedRunningTime="2026-07-09 04:27:24.965008117 +0000 UTC m=+1644.239386025"
Jul 09 04:27:25.109967 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.109929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.187693 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.187651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.196690 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.196660    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:25.200921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.200862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.203105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.203085    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f532f789-serving-cert\""
Jul 09 04:27:25.214089 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.214068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:25.232180 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.232159    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:25.236747 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.236724    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:25.262761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.262726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.320278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.320236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.320493 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.320311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.320493 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.320453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pthsq\" (UniqueName: \"kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.404783 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.404741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.422015 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.421984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pthsq\" (UniqueName: \"kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.422187 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.422040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.422187 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.422071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.422767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.422745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.424500 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.424448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.429895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.429870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pthsq\" (UniqueName: \"kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq\") pod \"maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.513014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.512929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:25.658877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.658852    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:25.660789 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:27:25.660767    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2eedbc_4622_45b0_977a_05591b44c181.slice/crio-7416b38ba908a7262e873d450d18ea17454c6367ff32b26b0e5ff2db88562af2 WatchSource:0}: Error finding container 7416b38ba908a7262e873d450d18ea17454c6367ff32b26b0e5ff2db88562af2: Status 404 returned error can't find the container with id 7416b38ba908a7262e873d450d18ea17454c6367ff32b26b0e5ff2db88562af2
Jul 09 04:27:25.662978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.662944    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.722502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.722445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:25.956116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.956080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" event={"ID":"9e2eedbc-4622-45b0-977a-05591b44c181","Type":"ContainerStarted","Data":"7416b38ba908a7262e873d450d18ea17454c6367ff32b26b0e5ff2db88562af2"}
Jul 09 04:27:25.956669 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.956457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:25.957498 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:25.957454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:26.963718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:26.963685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" event={"ID":"9e2eedbc-4622-45b0-977a-05591b44c181","Type":"ContainerStarted","Data":"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"}
Jul 09 04:27:26.964114 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:26.963832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:26.980334 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:26.980288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" podStartSLOduration=1.666430021 podStartE2EDuration="1.980274361s" totalImagesPullingTime="313.84434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:27:26.9776385 +0000 UTC m=+1646.252016422" watchObservedRunningTime="2026-07-09 04:27:26.980274361 +0000 UTC m=+1646.254652270"
Jul 09 04:27:27.260653 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:27.260601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:29.580006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:29.579973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:29.722979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:29.722947    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:29.723174 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:29.723153    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" podUID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" containerName="istio-proxy" containerID="cri-o://db1b3bd4977caca0b8811bd71b339acd7c2633b383fe79fa1387209995f806b8" gracePeriod=30
Jul 09 04:27:29.742456 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:29.742341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:29.843363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:29.843265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:30.005005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.004957    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:30.005562 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.005518    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" podUID="9e2eedbc-4622-45b0-977a-05591b44c181" containerName="maas-api" containerID="cri-o://7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251" gracePeriod=30
Jul 09 04:27:30.038490 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.034764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:30.289862 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.289838    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:30.471333 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.471249    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca\") pod \"9e2eedbc-4622-45b0-977a-05591b44c181\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") "
Jul 09 04:27:30.471530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.471370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls\") pod \"9e2eedbc-4622-45b0-977a-05591b44c181\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") "
Jul 09 04:27:30.471530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.471401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pthsq\" (UniqueName: \"kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq\") pod \"9e2eedbc-4622-45b0-977a-05591b44c181\" (UID: \"9e2eedbc-4622-45b0-977a-05591b44c181\") "
Jul 09 04:27:30.471626 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.471597    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e2eedbc-4622-45b0-977a-05591b44c181" (UID: "9e2eedbc-4622-45b0-977a-05591b44c181"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:27:30.473477 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.473437    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq" (OuterVolumeSpecName: "kube-api-access-pthsq") pod "9e2eedbc-4622-45b0-977a-05591b44c181" (UID: "9e2eedbc-4622-45b0-977a-05591b44c181"). InnerVolumeSpecName "kube-api-access-pthsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:30.473592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.473563    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e2eedbc-4622-45b0-977a-05591b44c181" (UID: "9e2eedbc-4622-45b0-977a-05591b44c181"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:27:30.572551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.572518    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2eedbc-4622-45b0-977a-05591b44c181-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:30.572551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.572545    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2eedbc-4622-45b0-977a-05591b44c181-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:30.572551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.572555    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pthsq\" (UniqueName: \"kubernetes.io/projected/9e2eedbc-4622-45b0-977a-05591b44c181-kube-api-access-pthsq\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:30.987767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.987731    2577 generic.go:356] "Generic (PLEG): container finished" podID="9e2eedbc-4622-45b0-977a-05591b44c181" containerID="7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251" exitCode=0
Jul 09 04:27:30.988190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.987778    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"
Jul 09 04:27:30.988190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.987811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" event={"ID":"9e2eedbc-4622-45b0-977a-05591b44c181","Type":"ContainerDied","Data":"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"}
Jul 09 04:27:30.988190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.987862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn" event={"ID":"9e2eedbc-4622-45b0-977a-05591b44c181","Type":"ContainerDied","Data":"7416b38ba908a7262e873d450d18ea17454c6367ff32b26b0e5ff2db88562af2"}
Jul 09 04:27:30.988190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.987883    2577 scope.go:117] "RemoveContainer" containerID="7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"
Jul 09 04:27:30.996790 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.996771    2577 scope.go:117] "RemoveContainer" containerID="7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"
Jul 09 04:27:30.997023 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:27:30.997004    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251\": container with ID starting with 7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251 not found: ID does not exist" containerID="7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"
Jul 09 04:27:30.997073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:30.997031    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251"} err="failed to get container status \"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251\": rpc error: code = NotFound desc = could not find container \"7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251\": container with ID starting with 7b1d00a83955b9b2befe315ab6460c99e949aa0f246a256ff2bfad08b1a08251 not found: ID does not exist"
Jul 09 04:27:31.008191 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:31.008169    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:31.011278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:31.011260    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f532f789-765ccfcbd-w6tgn"]
Jul 09 04:27:31.179656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:31.179625    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2eedbc-4622-45b0-977a-05591b44c181" path="/var/lib/kubelet/pods/9e2eedbc-4622-45b0-977a-05591b44c181/volumes"
Jul 09 04:27:31.510002 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:31.509976    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:35.708654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:35.708605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:35.808721 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:35.808673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:36.023422 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.023382    2577 generic.go:356] "Generic (PLEG): container finished" podID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" containerID="db1b3bd4977caca0b8811bd71b339acd7c2633b383fe79fa1387209995f806b8" exitCode=0
Jul 09 04:27:36.023631 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.023448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" event={"ID":"e93b6b65-cf2c-4cab-aa72-ed4432396f87","Type":"ContainerDied","Data":"db1b3bd4977caca0b8811bd71b339acd7c2633b383fe79fa1387209995f806b8"}
Jul 09 04:27:36.085814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.085791    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:36.116530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116495    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.116725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116543    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.116725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116586    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.116725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.116725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116691    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.116725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116718    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.117009 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116769    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.117009 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116794    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.117009 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfn8j\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j\") pod \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\" (UID: \"e93b6b65-cf2c-4cab-aa72-ed4432396f87\") "
Jul 09 04:27:36.117009 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.116969    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:36.117216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.117113    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:27:36.117269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.117225    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:36.117269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.117249    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.117373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.117270    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.117373 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.117318    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:36.118129 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.118098    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data" (OuterVolumeSpecName: "istio-data") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:36.118971 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.118944    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:27:36.119141 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.119122    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j" (OuterVolumeSpecName: "kube-api-access-bfn8j") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "kube-api-access-bfn8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:36.119242 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.119227    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:27:36.119294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.119226    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token" (OuterVolumeSpecName: "istio-token") pod "e93b6b65-cf2c-4cab-aa72-ed4432396f87" (UID: "e93b6b65-cf2c-4cab-aa72-ed4432396f87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:27:36.190140 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.190100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218373    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218405    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218422    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218434    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218448    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218461    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfn8j\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-kube-api-access-bfn8j\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.218842 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.218504    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93b6b65-cf2c-4cab-aa72-ed4432396f87-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:27:36.279212 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.279182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:36.382615 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:36.382581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:37.031261 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.031229    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"
Jul 09 04:27:37.031261 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.031253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz" event={"ID":"e93b6b65-cf2c-4cab-aa72-ed4432396f87","Type":"ContainerDied","Data":"3a81b4d72b79d298e93c249e8f30596a43e96c14b4be8062797ca1dd5ff2b4d2"}
Jul 09 04:27:37.031850 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.031327    2577 scope.go:117] "RemoveContainer" containerID="db1b3bd4977caca0b8811bd71b339acd7c2633b383fe79fa1387209995f806b8"
Jul 09 04:27:37.057814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.057791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:37.060195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.060176    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f532f789-openshift-default-566ddc4f79-b85tz"]
Jul 09 04:27:37.181291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.181258    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" path="/var/lib/kubelet/pods/e93b6b65-cf2c-4cab-aa72-ed4432396f87/volumes"
Jul 09 04:27:37.778890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:37.778858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:43.512633 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:43.512596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:43.614513 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:43.614458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:43.713109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:43.713078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:43.813745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:43.813666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:27:44.205876 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:27:44.205845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:01.065058 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:01.065029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:10.130310 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:10.130276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:15.940295 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:15.940264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:16.107151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:16.107114    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:16.202743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:16.202651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:32.057408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:32.057326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:37.958899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:37.958843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:38.023418 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:38.023393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:38.100012 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:38.099978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:38.208561 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:38.208527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:48.693852 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:48.693803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:49.493614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:49.493586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:55.142975 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:55.142941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:55.201992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:55.201961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:55.273240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:55.273209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:55.418109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:55.418028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:55.539601 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:55.539569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:56.212740 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:56.212708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:56.313169 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:56.313134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:56.413242 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:56.413207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:28:58.017179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:28:58.017143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:03.699742 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:03.699705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:03.800311 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:03.800282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:04.339283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:04.339240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:09.946065 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:09.946030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:10.046039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.046012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:10.203724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.203642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:10.512048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.511975    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:10.512448 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512435    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2eedbc-4622-45b0-977a-05591b44c181" containerName="maas-api"
Jul 09 04:29:10.512521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512450    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2eedbc-4622-45b0-977a-05591b44c181" containerName="maas-api"
Jul 09 04:29:10.512521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512462    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" containerName="istio-proxy"
Jul 09 04:29:10.512521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512499    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" containerName="istio-proxy"
Jul 09 04:29:10.512625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512584    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e93b6b65-cf2c-4cab-aa72-ed4432396f87" containerName="istio-proxy"
Jul 09 04:29:10.512625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.512593    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2eedbc-4622-45b0-977a-05591b44c181" containerName="maas-api"
Jul 09 04:29:10.515820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.515802    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.517762 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.517747    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a21ac036-openshift-default-dockercfg-hx99k\""
Jul 09 04:29:10.536120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.536096    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:10.545636 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.545610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:10.627654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.627821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.627821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.627821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.627821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.627821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.628094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvjn\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.628094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.628094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.627968    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.637395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.637366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:10.728482 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvjn\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.728864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.729076 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.728903    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.729136 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.729105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.729192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.729140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.729407 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.729382    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.729894 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.729874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.731572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.731542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.731662 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.731593    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.735989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.735966    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.736177 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.736151    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvjn\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn\") pod \"e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") " pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.825670 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.825610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:10.967549 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.967522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:10.969018 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:29:10.968992    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9584621_39e4_4dc8_94b8_64684d2928b8.slice/crio-e47bcb2b3baa117f98c67379cbc83533c1990e62fa555896f9c5320c326bb637 WatchSource:0}: Error finding container e47bcb2b3baa117f98c67379cbc83533c1990e62fa555896f9c5320c326bb637: Status 404 returned error can't find the container with id e47bcb2b3baa117f98c67379cbc83533c1990e62fa555896f9c5320c326bb637
Jul 09 04:29:10.971199 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.971167    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:10.971291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.971231    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:10.971291 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:10.971262    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:11.554409 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:11.554371    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerStarted","Data":"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"}
Jul 09 04:29:11.554600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:11.554426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerStarted","Data":"e47bcb2b3baa117f98c67379cbc83533c1990e62fa555896f9c5320c326bb637"}
Jul 09 04:29:11.573185 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:11.573142    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" podStartSLOduration=1.573128941 podStartE2EDuration="1.573128941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:29:11.570460318 +0000 UTC m=+1750.844838225" watchObservedRunningTime="2026-07-09 04:29:11.573128941 +0000 UTC m=+1750.847506852"
Jul 09 04:29:11.826306 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:11.826229    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:12.831266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:12.831235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:13.479417 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.479383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:13.557257 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.557227    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:13.561094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.561073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.562988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.562968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a21ac036-serving-cert\""
Jul 09 04:29:13.566679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.566651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:13.567824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.567807    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:13.573612 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.573594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:13.579665 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.579641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:13.645811 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.645780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:13.654146 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.654116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.654300 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.654283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhpwj\" (UniqueName: \"kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.654382 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.654360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.755759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.755677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhpwj\" (UniqueName: \"kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.755759 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.755735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.756000 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.755796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.756000 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:29:13.755914    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-serving-cert: secret "maas-api-e2e-mt-a21ac036-serving-cert" not found
Jul 09 04:29:13.756097 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:29:13.756024    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls podName:617d085a-baaa-4d68-affb-4bff32c496cc nodeName:}" failed. No retries permitted until 2026-07-09 04:29:14.256002444 +0000 UTC m=+1753.530380331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls") pod "maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" (UID: "617d085a-baaa-4d68-affb-4bff32c496cc") : secret "maas-api-e2e-mt-a21ac036-serving-cert" not found
Jul 09 04:29:13.756426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.756408    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.768810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.768791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhpwj\" (UniqueName: \"kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:13.836057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:13.836024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:14.128189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.128159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:14.232259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.232209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:14.262012 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.261970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:14.264745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.264718    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") pod \"maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:14.472793 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.472704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:14.615859 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:14.615833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:14.617703 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:29:14.617678    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617d085a_baaa_4d68_affb_4bff32c496cc.slice/crio-74a0ff51e8d35146fed9c14fecec8faf70aeab0de9d3c384aefec095770aac88 WatchSource:0}: Error finding container 74a0ff51e8d35146fed9c14fecec8faf70aeab0de9d3c384aefec095770aac88: Status 404 returned error can't find the container with id 74a0ff51e8d35146fed9c14fecec8faf70aeab0de9d3c384aefec095770aac88
Jul 09 04:29:15.583014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:15.582925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" event={"ID":"617d085a-baaa-4d68-affb-4bff32c496cc","Type":"ContainerStarted","Data":"54c317e7a57973493a2f27aa514f2db963bd4de39b62df603933ce0da9c11cc4"}
Jul 09 04:29:15.583014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:15.582986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" event={"ID":"617d085a-baaa-4d68-affb-4bff32c496cc","Type":"ContainerStarted","Data":"74a0ff51e8d35146fed9c14fecec8faf70aeab0de9d3c384aefec095770aac88"}
Jul 09 04:29:15.583449 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:15.583286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:15.599006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:15.598963    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" podStartSLOduration=1.971634104 podStartE2EDuration="2.598950236s" totalImagesPullingTime="627.316132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:29:15.596712146 +0000 UTC m=+1754.871090053" watchObservedRunningTime="2026-07-09 04:29:15.598950236 +0000 UTC m=+1754.873328183"
Jul 09 04:29:17.971037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:17.971009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:18.666805 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:18.666772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:18.791309 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:18.791279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:19.045196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:19.045121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:21.593701 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:21.593674    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:24.682501 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:24.682450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:24.748528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:24.748494    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:25.081816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.081702    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:25.447072 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.447033    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:25.447314 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.447283    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" podUID="617d085a-baaa-4d68-affb-4bff32c496cc" containerName="maas-api" containerID="cri-o://54c317e7a57973493a2f27aa514f2db963bd4de39b62df603933ce0da9c11cc4" gracePeriod=30
Jul 09 04:29:25.479011 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.478977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:25.580625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.580591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:25.651526 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.651268    2577 generic.go:356] "Generic (PLEG): container finished" podID="617d085a-baaa-4d68-affb-4bff32c496cc" containerID="54c317e7a57973493a2f27aa514f2db963bd4de39b62df603933ce0da9c11cc4" exitCode=0
Jul 09 04:29:25.651526 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.651362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" event={"ID":"617d085a-baaa-4d68-affb-4bff32c496cc","Type":"ContainerDied","Data":"54c317e7a57973493a2f27aa514f2db963bd4de39b62df603933ce0da9c11cc4"}
Jul 09 04:29:25.704035 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.703965    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:25.787789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.787759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca\") pod \"617d085a-baaa-4d68-affb-4bff32c496cc\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") "
Jul 09 04:29:25.787977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.787808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") pod \"617d085a-baaa-4d68-affb-4bff32c496cc\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") "
Jul 09 04:29:25.787977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.787839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhpwj\" (UniqueName: \"kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj\") pod \"617d085a-baaa-4d68-affb-4bff32c496cc\" (UID: \"617d085a-baaa-4d68-affb-4bff32c496cc\") "
Jul 09 04:29:25.788173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.788150    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "617d085a-baaa-4d68-affb-4bff32c496cc" (UID: "617d085a-baaa-4d68-affb-4bff32c496cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:29:25.789879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.789859    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "617d085a-baaa-4d68-affb-4bff32c496cc" (UID: "617d085a-baaa-4d68-affb-4bff32c496cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:29:25.790054 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.790031    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj" (OuterVolumeSpecName: "kube-api-access-vhpwj") pod "617d085a-baaa-4d68-affb-4bff32c496cc" (UID: "617d085a-baaa-4d68-affb-4bff32c496cc"). InnerVolumeSpecName "kube-api-access-vhpwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:29:25.889185 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.889150    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/617d085a-baaa-4d68-affb-4bff32c496cc-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:25.889185 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.889179    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/617d085a-baaa-4d68-affb-4bff32c496cc-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:25.889185 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:25.889190    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhpwj\" (UniqueName: \"kubernetes.io/projected/617d085a-baaa-4d68-affb-4bff32c496cc-kube-api-access-vhpwj\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:26.658047 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:26.658004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468" event={"ID":"617d085a-baaa-4d68-affb-4bff32c496cc","Type":"ContainerDied","Data":"74a0ff51e8d35146fed9c14fecec8faf70aeab0de9d3c384aefec095770aac88"}
Jul 09 04:29:26.658317 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:26.658060    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"
Jul 09 04:29:26.658317 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:26.658101    2577 scope.go:117] "RemoveContainer" containerID="54c317e7a57973493a2f27aa514f2db963bd4de39b62df603933ce0da9c11cc4"
Jul 09 04:29:26.682552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:26.682499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:26.686016 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:26.685994    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a21ac036-7d4b55c79f-sp468"]
Jul 09 04:29:27.180938 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.180856    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="617d085a-baaa-4d68-affb-4bff32c496cc" path="/var/lib/kubelet/pods/617d085a-baaa-4d68-affb-4bff32c496cc/volumes"
Jul 09 04:29:27.665972 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.665939    2577 generic.go:356] "Generic (PLEG): container finished" podID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerID="ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4" exitCode=0
Jul 09 04:29:27.666170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.666015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerDied","Data":"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"}
Jul 09 04:29:27.666457 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.666442    2577 scope.go:117] "RemoveContainer" containerID="ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"
Jul 09 04:29:27.667454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.667424    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:27.667564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.667504    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:27.667564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:27.667556    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:28.567095 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:28.567062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:28.674168 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:28.674133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerStarted","Data":"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"}
Jul 09 04:29:28.826837 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:28.826728    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:28.828417 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:28.828380    2577 patch_prober.go:28] interesting pod/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 09 04:29:28.828592 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:28.828443    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 09 04:29:29.826030 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:29.825996    2577 patch_prober.go:28] interesting pod/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 09 04:29:29.826443 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:29.826047    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 09 04:29:30.826375 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:30.826337    2577 patch_prober.go:28] interesting pod/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 09 04:29:30.826912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:30.826403    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 09 04:29:31.326160 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.326122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:31.484954 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.484920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:31.619089 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.619050    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:31.654493 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.649267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:31.691429 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.691391    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy" containerID="cri-o://a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf" gracePeriod=30
Jul 09 04:29:31.851459 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.851409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:31.985586 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:31.985457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:32.082618 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:32.082582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:32.198246 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:32.198214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:38.048702 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.048646    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:38.117689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117656    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.117689 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.117921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.117921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.117921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117780    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.117921 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xvjn\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.118137 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.117988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.118137 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118004    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:29:38.118137 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.118137 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo\") pod \"d9584621-39e4-4dc8-94b8-64684d2928b8\" (UID: \"d9584621-39e4-4dc8-94b8-64684d2928b8\") "
Jul 09 04:29:38.118341 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118314    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:29:38.118391 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118338    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:29:38.118484 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118445    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.118551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118493    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9584621-39e4-4dc8-94b8-64684d2928b8-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.118551 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118507    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.118680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118643    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:29:38.118743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.118702    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:29:38.120018 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.119999    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:29:38.120405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.120378    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:29:38.120518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.120435    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:29:38.120518 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.120443    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn" (OuterVolumeSpecName: "kube-api-access-7xvjn") pod "d9584621-39e4-4dc8-94b8-64684d2928b8" (UID: "d9584621-39e4-4dc8-94b8-64684d2928b8"). InnerVolumeSpecName "kube-api-access-7xvjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:29:38.134928 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.134903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:38.219454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219429    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.219454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219452    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.219454 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219460    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.219680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219495    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xvjn\" (UniqueName: \"kubernetes.io/projected/d9584621-39e4-4dc8-94b8-64684d2928b8-kube-api-access-7xvjn\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.219680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219506    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9584621-39e4-4dc8-94b8-64684d2928b8-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.219680 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.219515    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9584621-39e4-4dc8-94b8-64684d2928b8-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:29:38.241924 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.241888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:38.332634 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.332551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:38.474036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.474001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:38.732697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.732667    2577 generic.go:356] "Generic (PLEG): container finished" podID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerID="a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf" exitCode=0
Jul 09 04:29:38.732884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.732712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"
Jul 09 04:29:38.732884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.732759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerDied","Data":"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"}
Jul 09 04:29:38.732884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.732812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76" event={"ID":"d9584621-39e4-4dc8-94b8-64684d2928b8","Type":"ContainerDied","Data":"e47bcb2b3baa117f98c67379cbc83533c1990e62fa555896f9c5320c326bb637"}
Jul 09 04:29:38.732884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.732835    2577 scope.go:117] "RemoveContainer" containerID="a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"
Jul 09 04:29:38.743267 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.743251    2577 scope.go:117] "RemoveContainer" containerID="ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"
Jul 09 04:29:38.752135 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.752109    2577 scope.go:117] "RemoveContainer" containerID="a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"
Jul 09 04:29:38.752428 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:29:38.752407    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf\": container with ID starting with a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf not found: ID does not exist" containerID="a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"
Jul 09 04:29:38.752540 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.752435    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf"} err="failed to get container status \"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf\": rpc error: code = NotFound desc = could not find container \"a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf\": container with ID starting with a2164fcaf9027a10d3a95bf3976ef2f3c3accef6baec736b7b63117d65bad7cf not found: ID does not exist"
Jul 09 04:29:38.752540 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.752453    2577 scope.go:117] "RemoveContainer" containerID="ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"
Jul 09 04:29:38.752743 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:29:38.752723    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4\": container with ID starting with ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4 not found: ID does not exist" containerID="ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"
Jul 09 04:29:38.752795 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.752753    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4"} err="failed to get container status \"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4\": rpc error: code = NotFound desc = could not find container \"ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4\": container with ID starting with ebbb26b08ea16ea98641096479af5d3eef48f6caf1c632bc30451939289b08d4 not found: ID does not exist"
Jul 09 04:29:38.754792 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.754773    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:38.761761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:38.761739    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a21ac036-openshift-default-77c8c6f88c-9dc76"]
Jul 09 04:29:39.180764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:39.180728    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" path="/var/lib/kubelet/pods/d9584621-39e4-4dc8-94b8-64684d2928b8/volumes"
Jul 09 04:29:44.397070 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:44.397029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:44.465681 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:44.465644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:51.183561 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.183511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:51.459359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459268    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:29:51.459887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459858    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.459887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459881    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.459887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459893    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459918    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617d085a-baaa-4d68-affb-4bff32c496cc" containerName="maas-api"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459923    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="617d085a-baaa-4d68-affb-4bff32c496cc" containerName="maas-api"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459988    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.459998    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="617d085a-baaa-4d68-affb-4bff32c496cc" containerName="maas-api"
Jul 09 04:29:51.460126 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.460004    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9584621-39e4-4dc8-94b8-64684d2928b8" containerName="istio-proxy"
Jul 09 04:29:51.464540 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.464519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.466675 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.466636    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e747c320-openshift-default-dockercfg-mf6ck\""
Jul 09 04:29:51.475498 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.475457    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:29:51.495560 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.495525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:51.651614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651614 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2cbm\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.651853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.652082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.652082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.651966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2cbm\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753723 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753723 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753826 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753730    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.753924 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753899    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.754109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.754109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753937    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.754226 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.753981    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.756222 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.756197    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.756428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.756411    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.766608 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.766585    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.768584 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.768558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2cbm\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm\") pod \"e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") " pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.778312 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.778281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:51.783789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.783761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:51.932966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.932940    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:29:51.935189 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:29:51.935164    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f28d312_c300_44f0_8e8a_677f58463fb9.slice/crio-7a555b2e0334b9ac412585b4643dfc5f999bf6eb78b3879d715a968ab991b640 WatchSource:0}: Error finding container 7a555b2e0334b9ac412585b4643dfc5f999bf6eb78b3879d715a968ab991b640: Status 404 returned error can't find the container with id 7a555b2e0334b9ac412585b4643dfc5f999bf6eb78b3879d715a968ab991b640
Jul 09 04:29:51.937374 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.937345    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:51.937439 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.937406    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:51.937510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:51.937443    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:29:52.811895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:52.811817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerStarted","Data":"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"}
Jul 09 04:29:52.811895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:52.811869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerStarted","Data":"7a555b2e0334b9ac412585b4643dfc5f999bf6eb78b3879d715a968ab991b640"}
Jul 09 04:29:52.832412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:52.832358    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podStartSLOduration=1.832336376 podStartE2EDuration="1.832336376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:29:52.831093368 +0000 UTC m=+1792.105471288" watchObservedRunningTime="2026-07-09 04:29:52.832336376 +0000 UTC m=+1792.106714285"
Jul 09 04:29:53.779399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:53.779361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:53.784362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:53.784340    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:53.818216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:53.818189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:53.819214 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:53.819195    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:29:54.720720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.720683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:54.790838 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.790793    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:29:54.795120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.795084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:54.797030 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.797008    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e747c320-serving-cert\""
Jul 09 04:29:54.807709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.807681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:29:54.811530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.811504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:54.870742 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.870708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:54.972221 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.972116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:54.983513 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.983463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g98v\" (UniqueName: \"kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:54.983738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.983555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:54.983738 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:54.983611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.084711 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.084672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g98v\" (UniqueName: \"kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.084873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.084729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.084873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.084768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.085441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.085418    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.087260 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.087238    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.092716 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.092683    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g98v\" (UniqueName: \"kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v\") pod \"maas-api-e2e-mt-e747c320-8544d4665c-89gjr\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.107417 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.107377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.261676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.261618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:29:55.264028 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:29:55.263995    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda324b421_5d41_4c95_ade3_1a973b13a427.slice/crio-9d1ff1f7459b2fa431ab4abb6ad16af6afba1a4b0cbf914f5648e27c0a2ddac7 WatchSource:0}: Error finding container 9d1ff1f7459b2fa431ab4abb6ad16af6afba1a4b0cbf914f5648e27c0a2ddac7: Status 404 returned error can't find the container with id 9d1ff1f7459b2fa431ab4abb6ad16af6afba1a4b0cbf914f5648e27c0a2ddac7
Jul 09 04:29:55.404772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.404732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:55.501347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.501315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:29:55.835806 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.835757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" event={"ID":"a324b421-5d41-4c95-ade3-1a973b13a427","Type":"ContainerStarted","Data":"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"}
Jul 09 04:29:55.835996 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.835864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:29:55.835996 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.835895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" event={"ID":"a324b421-5d41-4c95-ade3-1a973b13a427","Type":"ContainerStarted","Data":"9d1ff1f7459b2fa431ab4abb6ad16af6afba1a4b0cbf914f5648e27c0a2ddac7"}
Jul 09 04:29:55.855318 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:55.855261    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" podStartSLOduration=1.436632103 podStartE2EDuration="1.855244592s" totalImagesPullingTime="418.612489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:29:55.853712037 +0000 UTC m=+1795.128089948" watchObservedRunningTime="2026-07-09 04:29:55.855244592 +0000 UTC m=+1795.129622500"
Jul 09 04:29:59.217412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:29:59.217375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:01.242130 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:01.242101    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:30:01.252298 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:01.252263    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:30:01.854849 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:01.854820    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:30:08.819804 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.819721    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 09 04:30:08.819804 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.819781    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 09 04:30:08.916644 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.916615    2577 generic.go:356] "Generic (PLEG): container finished" podID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerID="158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda" exitCode=0
Jul 09 04:30:08.916772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.916687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerDied","Data":"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"}
Jul 09 04:30:08.917066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.917052    2577 scope.go:117] "RemoveContainer" containerID="158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"
Jul 09 04:30:08.918042 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.918013    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:30:08.918114 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.918097    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:30:08.918164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:08.918148    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:30:09.925480 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:09.925443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerStarted","Data":"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"}
Jul 09 04:30:10.779336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:10.779299    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:30:10.779618 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:10.779597    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 09 04:30:10.779671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:10.779645    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 09 04:30:11.741555 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:11.741499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:11.778875 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:11.778842    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 09 04:30:11.779036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:11.778907    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 09 04:30:12.779322 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:12.779286    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 09 04:30:12.779729 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:12.779335    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 09 04:30:13.779624 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:13.779586    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 09 04:30:13.780031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:13.779654    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 09 04:30:14.782764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:14.782732    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:30:14.783152 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:14.782995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:30:14.783830 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:14.783812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:30:19.080616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:19.080573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:19.179327 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:19.179294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:31.525813 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:31.525775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:32.194039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.194010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:32.290309 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.290276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:32.425623 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.425583    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:30:32.425883 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.425836    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" podUID="a324b421-5d41-4c95-ade3-1a973b13a427" containerName="maas-api" containerID="cri-o://56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158" gracePeriod=30
Jul 09 04:30:32.446359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.446328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:32.685377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.685356    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:30:32.859676 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.859643    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca\") pod \"a324b421-5d41-4c95-ade3-1a973b13a427\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") "
Jul 09 04:30:32.859824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.859716    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls\") pod \"a324b421-5d41-4c95-ade3-1a973b13a427\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") "
Jul 09 04:30:32.859824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.859776    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g98v\" (UniqueName: \"kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v\") pod \"a324b421-5d41-4c95-ade3-1a973b13a427\" (UID: \"a324b421-5d41-4c95-ade3-1a973b13a427\") "
Jul 09 04:30:32.860099 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.860074    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a324b421-5d41-4c95-ade3-1a973b13a427" (UID: "a324b421-5d41-4c95-ade3-1a973b13a427"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:30:32.860163 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.860139    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a324b421-5d41-4c95-ade3-1a973b13a427-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:30:32.861873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.861845    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v" (OuterVolumeSpecName: "kube-api-access-9g98v") pod "a324b421-5d41-4c95-ade3-1a973b13a427" (UID: "a324b421-5d41-4c95-ade3-1a973b13a427"). InnerVolumeSpecName "kube-api-access-9g98v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:30:32.861988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.861850    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a324b421-5d41-4c95-ade3-1a973b13a427" (UID: "a324b421-5d41-4c95-ade3-1a973b13a427"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:30:32.961565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.961534    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g98v\" (UniqueName: \"kubernetes.io/projected/a324b421-5d41-4c95-ade3-1a973b13a427-kube-api-access-9g98v\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:30:32.961565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:32.961562    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a324b421-5d41-4c95-ade3-1a973b13a427-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:30:33.097577 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.097544    2577 generic.go:356] "Generic (PLEG): container finished" podID="a324b421-5d41-4c95-ade3-1a973b13a427" containerID="56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158" exitCode=0
Jul 09 04:30:33.097772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.097589    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"
Jul 09 04:30:33.097772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.097636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" event={"ID":"a324b421-5d41-4c95-ade3-1a973b13a427","Type":"ContainerDied","Data":"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"}
Jul 09 04:30:33.097772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.097691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr" event={"ID":"a324b421-5d41-4c95-ade3-1a973b13a427","Type":"ContainerDied","Data":"9d1ff1f7459b2fa431ab4abb6ad16af6afba1a4b0cbf914f5648e27c0a2ddac7"}
Jul 09 04:30:33.097772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.097720    2577 scope.go:117] "RemoveContainer" containerID="56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"
Jul 09 04:30:33.107271 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.107250    2577 scope.go:117] "RemoveContainer" containerID="56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"
Jul 09 04:30:33.107566 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:30:33.107548    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158\": container with ID starting with 56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158 not found: ID does not exist" containerID="56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"
Jul 09 04:30:33.107639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.107578    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158"} err="failed to get container status \"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158\": rpc error: code = NotFound desc = could not find container \"56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158\": container with ID starting with 56dba2dfa3ce4ca2721dae5e93a8da26848c136b6166f15d787a5fb2b81da158 not found: ID does not exist"
Jul 09 04:30:33.118183 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.118133    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:30:33.123193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.123172    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e747c320-8544d4665c-89gjr"]
Jul 09 04:30:33.180337 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:33.180306    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a324b421-5d41-4c95-ade3-1a973b13a427" path="/var/lib/kubelet/pods/a324b421-5d41-4c95-ade3-1a973b13a427/volumes"
Jul 09 04:30:37.678708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:37.678660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:30:42.714735 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:30:42.714704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:18.810369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:18.810332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:19.861048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:19.861017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:25.327824 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.327746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:25.427688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.427649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:25.573912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.573879    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:31:25.574165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.574141    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" containerID="cri-o://8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4" gracePeriod=30
Jul 09 04:31:25.595168 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.595075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:25.813690 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:25.813654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:26.850100 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:26.850052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:26.942219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:26.942192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:27.039173 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:27.039127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:27.743944 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:27.743912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:27.840069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:27.840022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:27.941240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:27.941210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:29.223581 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:29.223530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:29.785630 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:29.785599    2577 patch_prober.go:28] interesting pod/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 04:31:29.785799 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:29.785654    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 04:31:30.836953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.836886    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:31:30.889617 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889589    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889755 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889711    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889833 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889833 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889816    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.889912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889882    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:31:30.889912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889900    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.890116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.889934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2cbm\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm\") pod \"7f28d312-c300-44f0-8e8a-677f58463fb9\" (UID: \"7f28d312-c300-44f0-8e8a-677f58463fb9\") "
Jul 09 04:31:30.890116 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890032    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:31:30.890221 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890143    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:31:30.890345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890315    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.890345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890340    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.890457 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890359    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.890457 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890440    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:31:30.890664 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.890633    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:31:30.892054 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.892005    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:31:30.892054 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.892041    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:31:30.892184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.892088    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:31:30.892184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.892114    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm" (OuterVolumeSpecName: "kube-api-access-q2cbm") pod "7f28d312-c300-44f0-8e8a-677f58463fb9" (UID: "7f28d312-c300-44f0-8e8a-677f58463fb9"). InnerVolumeSpecName "kube-api-access-q2cbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:31:30.991642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991614    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.991642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991639    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.991642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991647    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.991642 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991656    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f28d312-c300-44f0-8e8a-677f58463fb9-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.991900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991665    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2cbm\" (UniqueName: \"kubernetes.io/projected/7f28d312-c300-44f0-8e8a-677f58463fb9-kube-api-access-q2cbm\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:30.991900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:30.991673    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f28d312-c300-44f0-8e8a-677f58463fb9-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:31:31.421903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.421864    2577 generic.go:356] "Generic (PLEG): container finished" podID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerID="8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4" exitCode=0
Jul 09 04:31:31.422053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.421901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerDied","Data":"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"}
Jul 09 04:31:31.422053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.421932    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"
Jul 09 04:31:31.422053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.421958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j" event={"ID":"7f28d312-c300-44f0-8e8a-677f58463fb9","Type":"ContainerDied","Data":"7a555b2e0334b9ac412585b4643dfc5f999bf6eb78b3879d715a968ab991b640"}
Jul 09 04:31:31.422053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.421981    2577 scope.go:117] "RemoveContainer" containerID="8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"
Jul 09 04:31:31.431579 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.431550    2577 scope.go:117] "RemoveContainer" containerID="158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"
Jul 09 04:31:31.439342 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.439275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:31:31.441208 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.441188    2577 scope.go:117] "RemoveContainer" containerID="8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"
Jul 09 04:31:31.441483 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:31:31.441440    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4\": container with ID starting with 8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4 not found: ID does not exist" containerID="8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"
Jul 09 04:31:31.441576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.441501    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4"} err="failed to get container status \"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4\": rpc error: code = NotFound desc = could not find container \"8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4\": container with ID starting with 8486fc3b16a462a79abebdb76c75d777951e4de895d3ea09e3b6ed66dd3aa3b4 not found: ID does not exist"
Jul 09 04:31:31.441576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.441526    2577 scope.go:117] "RemoveContainer" containerID="158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"
Jul 09 04:31:31.441773 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:31:31.441755    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda\": container with ID starting with 158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda not found: ID does not exist" containerID="158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"
Jul 09 04:31:31.441821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.441781    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda"} err="failed to get container status \"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda\": rpc error: code = NotFound desc = could not find container \"158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda\": container with ID starting with 158274f33c961a9a57396abfa6f8bba0687a9355f0b7748b8d27bf24588d8dda not found: ID does not exist"
Jul 09 04:31:31.442660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:31.442643    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e747c320-openshift-default-84fc75b995-f2j2j"]
Jul 09 04:31:33.181546 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:33.181506    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" path="/var/lib/kubelet/pods/7f28d312-c300-44f0-8e8a-677f58463fb9/volumes"
Jul 09 04:31:34.951295 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:34.951259    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:35.052036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:35.052002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:35.604957 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:35.604925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:41.311588 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:41.311550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:41.413517 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:41.413464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:41.511671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:41.511601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:52.004947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:52.004915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:31:54.293197 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:31:54.293160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:13.241129 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:13.241100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:18.785165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:18.785128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:18.883563 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:18.883530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:19.113891 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:19.113864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:19.393329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:19.393254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:19.650785 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:19.650708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:22.631640 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.631600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:22.704619 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.704586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:22.705648 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.705626    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:32:22.706032 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:22.706092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706033    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:22.706092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706040    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a324b421-5d41-4c95-ade3-1a973b13a427" containerName="maas-api"
Jul 09 04:32:22.706092 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706045    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a324b421-5d41-4c95-ade3-1a973b13a427" containerName="maas-api"
Jul 09 04:32:22.706195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a324b421-5d41-4c95-ade3-1a973b13a427" containerName="maas-api"
Jul 09 04:32:22.706195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706149    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:22.706195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.706156    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:22.709396 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.709380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.711219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.711200    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c184fc-serving-cert\""
Jul 09 04:32:22.717254 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.717233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:32:22.738281 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.738249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.738408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.738297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lllfr\" (UniqueName: \"kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.738408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.738328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.774325 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.774296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:22.838802 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.838777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.839034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.838834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lllfr\" (UniqueName: \"kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.839034 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.838864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.839034 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:32:22.838916    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-serving-cert: secret "maas-api-e2e-inf-a-c184fc-serving-cert" not found
Jul 09 04:32:22.839034 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:32:22.838969    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls podName:0419b0d1-28f3-477f-90df-b1f8c684a941 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:23.338953345 +0000 UTC m=+1942.613331234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls") pod "maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" (UID: "0419b0d1-28f3-477f-90df-b1f8c684a941") : secret "maas-api-e2e-inf-a-c184fc-serving-cert" not found
Jul 09 04:32:22.839655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.839631    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:22.848768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:22.848747    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lllfr\" (UniqueName: \"kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:23.078107 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.078031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:23.299622 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.299584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:23.345568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.345433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:23.347977 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.347955    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") pod \"maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:23.400563 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.400524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:23.623202 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.623165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:23.768296 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.768271    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:32:23.770393 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:32:23.770363    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0419b0d1_28f3_477f_90df_b1f8c684a941.slice/crio-f34f7fb2579b3d0389b2f8e9f55c5f208f11f37881d082e329bdceb3e632a4b3 WatchSource:0}: Error finding container f34f7fb2579b3d0389b2f8e9f55c5f208f11f37881d082e329bdceb3e632a4b3: Status 404 returned error can't find the container with id f34f7fb2579b3d0389b2f8e9f55c5f208f11f37881d082e329bdceb3e632a4b3
Jul 09 04:32:23.771857 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:23.771839    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:32:24.724278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:24.724246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" event={"ID":"0419b0d1-28f3-477f-90df-b1f8c684a941","Type":"ContainerStarted","Data":"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"}
Jul 09 04:32:24.724278 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:24.724298    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:24.724593 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:24.724313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" event={"ID":"0419b0d1-28f3-477f-90df-b1f8c684a941","Type":"ContainerStarted","Data":"f34f7fb2579b3d0389b2f8e9f55c5f208f11f37881d082e329bdceb3e632a4b3"}
Jul 09 04:32:24.743877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:24.743830    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" podStartSLOduration=2.239021241 podStartE2EDuration="2.743813984s" totalImagesPullingTime="504.792743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:24.740759009 +0000 UTC m=+1944.015136918" watchObservedRunningTime="2026-07-09 04:32:24.743813984 +0000 UTC m=+1944.018191920"
Jul 09 04:32:27.145353 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.145320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:27.451976 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.451902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:27.723500 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.723409    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:32:27.723884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.723864    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:27.723884 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.723881    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f28d312-c300-44f0-8e8a-677f58463fb9" containerName="istio-proxy"
Jul 09 04:32:27.728303 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.727439    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.729799 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.729772    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b3e4d0-openshift-default-dockercfg-2lhwz\""
Jul 09 04:32:27.739498 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.739459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:32:27.759987 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.759961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:27.784332 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.784616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.784592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz664\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.885827 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.885827 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.885965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz664\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886303    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886684 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886649    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886684 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886664    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886696    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.886832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.886745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.888005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.887988    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.888361 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.888344    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.893168 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.893144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.893527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.893503    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz664\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664\") pod \"e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") " pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:27.900564 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:27.900524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:28.043161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.043077    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:28.190052 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.190028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:32:28.192768 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:32:28.192738    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9451c1d8_159a_43da_a6f9_1f6b1fe5e873.slice/crio-5e0dda27b35c500933b77591eeaefc738ce67723d2a8102838284a7deb63ef23 WatchSource:0}: Error finding container 5e0dda27b35c500933b77591eeaefc738ce67723d2a8102838284a7deb63ef23: Status 404 returned error can't find the container with id 5e0dda27b35c500933b77591eeaefc738ce67723d2a8102838284a7deb63ef23
Jul 09 04:32:28.194828 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.194798    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:28.194923 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.194859    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:28.194923 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.194893    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:28.750343 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.750310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerStarted","Data":"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"}
Jul 09 04:32:28.750534 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.750392    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerStarted","Data":"5e0dda27b35c500933b77591eeaefc738ce67723d2a8102838284a7deb63ef23"}
Jul 09 04:32:28.769189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:28.769145    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podStartSLOduration=1.76913141 podStartE2EDuration="1.76913141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:28.766265186 +0000 UTC m=+1948.040643105" watchObservedRunningTime="2026-07-09 04:32:28.76913141 +0000 UTC m=+1948.043509319"
Jul 09 04:32:29.043842 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:29.043769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:29.048165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:29.048143    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:29.755732 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:29.755703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:29.756568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:29.756547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:30.635618 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.635589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:30.706798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.706765    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:32:30.710510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.710490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.712586 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.712563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b3e4d0-serving-cert\""
Jul 09 04:32:30.714778 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.714757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:30.720593 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.720298    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:32:30.738836 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.738816    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:32:30.790983 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.790959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:30.810985 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.810959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tc2f\" (UniqueName: \"kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.811121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.811029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.811121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.811066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.912432 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.912346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tc2f\" (UniqueName: \"kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.912637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.912438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.912637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.912480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.912637 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:32:30.912570    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-serving-cert: secret "maas-api-e2e-inf-b-b3e4d0-serving-cert" not found
Jul 09 04:32:30.912826 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:32:30.912645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls podName:f8b72d78-c835-4b2a-80d5-08b21c546e65 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:31.412622916 +0000 UTC m=+1950.687000815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls") pod "maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" (UID: "f8b72d78-c835-4b2a-80d5-08b21c546e65") : secret "maas-api-e2e-inf-b-b3e4d0-serving-cert" not found
Jul 09 04:32:30.913164 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.913142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.919819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.919797    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tc2f\" (UniqueName: \"kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:30.962261 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:30.962235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:31.281529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.281421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:31.396254 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.396223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:31.416789 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.416756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:31.419038 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.419017    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") pod \"maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:31.624833 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.624802    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:31.774925 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:31.774899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:32:31.776247 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:32:31.776216    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b72d78_c835_4b2a_80d5_08b21c546e65.slice/crio-635de52c4c03bd3cb2d02994c0dd94ed9025ed3cf6510d4d7206f8c97b54f6fa WatchSource:0}: Error finding container 635de52c4c03bd3cb2d02994c0dd94ed9025ed3cf6510d4d7206f8c97b54f6fa: Status 404 returned error can't find the container with id 635de52c4c03bd3cb2d02994c0dd94ed9025ed3cf6510d4d7206f8c97b54f6fa
Jul 09 04:32:32.773610 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:32.773577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" event={"ID":"f8b72d78-c835-4b2a-80d5-08b21c546e65","Type":"ContainerStarted","Data":"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"}
Jul 09 04:32:32.773979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:32.773655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" event={"ID":"f8b72d78-c835-4b2a-80d5-08b21c546e65","Type":"ContainerStarted","Data":"635de52c4c03bd3cb2d02994c0dd94ed9025ed3cf6510d4d7206f8c97b54f6fa"}
Jul 09 04:32:32.773979 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:32.773732    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:32.789019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:32.788981    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" podStartSLOduration=2.344982803 podStartE2EDuration="2.788969422s" totalImagesPullingTime="443.986619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:32.787531248 +0000 UTC m=+1952.061909156" watchObservedRunningTime="2026-07-09 04:32:32.788969422 +0000 UTC m=+1952.063347332"
Jul 09 04:32:35.115262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:35.115225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:38.784205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:38.784177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:32:43.506120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:43.506082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:44.990446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:44.989497    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": read tcp 10.134.0.2:33916->10.134.0.74:15021: read: connection reset by peer" start-of-body=
Jul 09 04:32:44.990446 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:44.989578    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": read tcp 10.134.0.2:33916->10.134.0.74:15021: read: connection reset by peer"
Jul 09 04:32:45.610369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.610341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:45.661811 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.661781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:45.850705 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.850666    2577 generic.go:356] "Generic (PLEG): container finished" podID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerID="b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4" exitCode=0
Jul 09 04:32:45.850865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.850742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerDied","Data":"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"}
Jul 09 04:32:45.851155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.851140    2577 scope.go:117] "RemoveContainer" containerID="b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"
Jul 09 04:32:45.852188 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.852155    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:45.852263 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.852234    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:45.852309 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.852270    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:32:45.864909 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.864891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:45.966001 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:45.965949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:46.859074 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:46.859043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerStarted","Data":"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"}
Jul 09 04:32:47.044108 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:47.044077    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:47.044392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:47.044376    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:47.044444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:47.044417    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:48.043954 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:48.043914    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:48.044418 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:48.043978    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:49.044611 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:49.044575    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:49.045732 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:49.045686    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:50.044975 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:50.044930    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:50.045486 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:50.045007    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:51.043652 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:51.043607    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:51.043863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:51.043675    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:52.044323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:52.043839    2577 patch_prober.go:28] interesting pod/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 09 04:32:52.044323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:52.043934    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 09 04:32:53.047306 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:53.047227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:53.047774 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:53.047648    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:53.048232 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:53.048215    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:32:53.173367 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:53.173326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:32:53.274612 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:32:53.274586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:02.581244 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:02.581215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:02.685997 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:02.685948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:04.985435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:04.985402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:23.154390 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:23.154354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:24.130272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:24.130217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:24.278761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:24.278728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:24.792312 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:24.792282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:24.986266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:24.986230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:25.088155 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.088054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:25.212775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.212745    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:33:25.213184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.213152    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" podUID="0419b0d1-28f3-477f-90df-b1f8c684a941" containerName="maas-api" containerID="cri-o://216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02" gracePeriod=30
Jul 09 04:33:25.240218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.240196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:33:25.474038 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.474016    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:33:25.529188 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.529157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lllfr\" (UniqueName: \"kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr\") pod \"0419b0d1-28f3-477f-90df-b1f8c684a941\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") "
Jul 09 04:33:25.529398 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.529201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") pod \"0419b0d1-28f3-477f-90df-b1f8c684a941\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") "
Jul 09 04:33:25.529398 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.529249    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca\") pod \"0419b0d1-28f3-477f-90df-b1f8c684a941\" (UID: \"0419b0d1-28f3-477f-90df-b1f8c684a941\") "
Jul 09 04:33:25.529757 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.529729    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0419b0d1-28f3-477f-90df-b1f8c684a941" (UID: "0419b0d1-28f3-477f-90df-b1f8c684a941"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:33:25.531389 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.531364    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr" (OuterVolumeSpecName: "kube-api-access-lllfr") pod "0419b0d1-28f3-477f-90df-b1f8c684a941" (UID: "0419b0d1-28f3-477f-90df-b1f8c684a941"). InnerVolumeSpecName "kube-api-access-lllfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:33:25.531660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.531639    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0419b0d1-28f3-477f-90df-b1f8c684a941" (UID: "0419b0d1-28f3-477f-90df-b1f8c684a941"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:33:25.630768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.630722    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0419b0d1-28f3-477f-90df-b1f8c684a941-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:33:25.630768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.630763    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0419b0d1-28f3-477f-90df-b1f8c684a941-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:33:25.630768 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:25.630777    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lllfr\" (UniqueName: \"kubernetes.io/projected/0419b0d1-28f3-477f-90df-b1f8c684a941-kube-api-access-lllfr\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:33:26.103973 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.103935    2577 generic.go:356] "Generic (PLEG): container finished" podID="0419b0d1-28f3-477f-90df-b1f8c684a941" containerID="216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02" exitCode=0
Jul 09 04:33:26.104151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.103985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"
Jul 09 04:33:26.104151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.104018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" event={"ID":"0419b0d1-28f3-477f-90df-b1f8c684a941","Type":"ContainerDied","Data":"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"}
Jul 09 04:33:26.104151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.104071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng" event={"ID":"0419b0d1-28f3-477f-90df-b1f8c684a941","Type":"ContainerDied","Data":"f34f7fb2579b3d0389b2f8e9f55c5f208f11f37881d082e329bdceb3e632a4b3"}
Jul 09 04:33:26.104151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.104093    2577 scope.go:117] "RemoveContainer" containerID="216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"
Jul 09 04:33:26.114400 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.114383    2577 scope.go:117] "RemoveContainer" containerID="216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"
Jul 09 04:33:26.114719 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:33:26.114699    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02\": container with ID starting with 216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02 not found: ID does not exist" containerID="216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"
Jul 09 04:33:26.114794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.114731    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02"} err="failed to get container status \"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02\": rpc error: code = NotFound desc = could not find container \"216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02\": container with ID starting with 216b831de577bb45591c3581bb354268e139b7e0fe1d364298f5eb7fa0a4ab02 not found: ID does not exist"
Jul 09 04:33:26.125553 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.125522    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:33:26.129515 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:26.129488    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c184fc-68d88dd447-p5sng"]
Jul 09 04:33:27.180110 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:27.180079    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0419b0d1-28f3-477f-90df-b1f8c684a941" path="/var/lib/kubelet/pods/0419b0d1-28f3-477f-90df-b1f8c684a941/volumes"
Jul 09 04:33:29.797486 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:33:29.797428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:06.356656 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:06.356621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:07.440658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.440623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:07.537182 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.537149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:07.638412 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.638371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:07.710655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.710555    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:34:07.710843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.710810    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" podUID="f8b72d78-c835-4b2a-80d5-08b21c546e65" containerName="maas-api" containerID="cri-o://5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af" gracePeriod=30
Jul 09 04:34:07.738019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:07.737982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:08.175444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.175419    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:34:08.235519 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.235394    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tc2f\" (UniqueName: \"kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f\") pod \"f8b72d78-c835-4b2a-80d5-08b21c546e65\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") "
Jul 09 04:34:08.235519 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.235456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") pod \"f8b72d78-c835-4b2a-80d5-08b21c546e65\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") "
Jul 09 04:34:08.235753 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.235521    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca\") pod \"f8b72d78-c835-4b2a-80d5-08b21c546e65\" (UID: \"f8b72d78-c835-4b2a-80d5-08b21c546e65\") "
Jul 09 04:34:08.235894 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.235865    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8b72d78-c835-4b2a-80d5-08b21c546e65" (UID: "f8b72d78-c835-4b2a-80d5-08b21c546e65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:34:08.237598 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.237575    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8b72d78-c835-4b2a-80d5-08b21c546e65" (UID: "f8b72d78-c835-4b2a-80d5-08b21c546e65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:34:08.237882 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.237863    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f" (OuterVolumeSpecName: "kube-api-access-8tc2f") pod "f8b72d78-c835-4b2a-80d5-08b21c546e65" (UID: "f8b72d78-c835-4b2a-80d5-08b21c546e65"). InnerVolumeSpecName "kube-api-access-8tc2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:08.336832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.336804    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tc2f\" (UniqueName: \"kubernetes.io/projected/f8b72d78-c835-4b2a-80d5-08b21c546e65-kube-api-access-8tc2f\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:08.336832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.336832    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8b72d78-c835-4b2a-80d5-08b21c546e65-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:08.336832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.336841    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8b72d78-c835-4b2a-80d5-08b21c546e65-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:08.343200 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.343172    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8b72d78-c835-4b2a-80d5-08b21c546e65" containerID="5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af" exitCode=0
Jul 09 04:34:08.343351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.343217    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"
Jul 09 04:34:08.343351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.343248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" event={"ID":"f8b72d78-c835-4b2a-80d5-08b21c546e65","Type":"ContainerDied","Data":"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"}
Jul 09 04:34:08.343351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.343299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d" event={"ID":"f8b72d78-c835-4b2a-80d5-08b21c546e65","Type":"ContainerDied","Data":"635de52c4c03bd3cb2d02994c0dd94ed9025ed3cf6510d4d7206f8c97b54f6fa"}
Jul 09 04:34:08.343351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.343323    2577 scope.go:117] "RemoveContainer" containerID="5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"
Jul 09 04:34:08.357959 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.357940    2577 scope.go:117] "RemoveContainer" containerID="5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"
Jul 09 04:34:08.358216 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:08.358194    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af\": container with ID starting with 5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af not found: ID does not exist" containerID="5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"
Jul 09 04:34:08.358267 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.358225    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af"} err="failed to get container status \"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af\": rpc error: code = NotFound desc = could not find container \"5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af\": container with ID starting with 5cd47229770a0a79f1d88f4a6ae5795c2f0df0fd49ff002176aafed855c911af not found: ID does not exist"
Jul 09 04:34:08.365955 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.365932    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:34:08.368335 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:08.368316    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b3e4d0-77d548d749-7cp5d"]
Jul 09 04:34:09.180066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:09.180025    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8b72d78-c835-4b2a-80d5-08b21c546e65" path="/var/lib/kubelet/pods/f8b72d78-c835-4b2a-80d5-08b21c546e65/volumes"
Jul 09 04:34:13.321500 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.321453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:13.423405 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.423373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:13.545103 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.545072    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:34:13.545329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.545300    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy" containerID="cri-o://e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15" gracePeriod=30
Jul 09 04:34:13.560618 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.560588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:13.780275 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:13.780248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:14.107540 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:14.107507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:14.412297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:14.412215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:14.782820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:14.782735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:17.292947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.292916    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:17.293352 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293337    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8b72d78-c835-4b2a-80d5-08b21c546e65" containerName="maas-api"
Jul 09 04:34:17.293404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293353    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8b72d78-c835-4b2a-80d5-08b21c546e65" containerName="maas-api"
Jul 09 04:34:17.293404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293371    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0419b0d1-28f3-477f-90df-b1f8c684a941" containerName="maas-api"
Jul 09 04:34:17.293404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293379    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0419b0d1-28f3-477f-90df-b1f8c684a941" containerName="maas-api"
Jul 09 04:34:17.293536 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293479    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0419b0d1-28f3-477f-90df-b1f8c684a941" containerName="maas-api"
Jul 09 04:34:17.293536 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.293488    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8b72d78-c835-4b2a-80d5-08b21c546e65" containerName="maas-api"
Jul 09 04:34:17.297733 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.297713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.300038 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.300021    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-dac2f3-serving-cert\""
Jul 09 04:34:17.305048 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.305023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:17.428084 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.428053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.428270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.428132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s2pz\" (UniqueName: \"kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.428270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.428166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.529223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.529189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.529414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.529255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s2pz\" (UniqueName: \"kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.529414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.529290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.529578 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:17.529416    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-serving-cert: secret "maas-api-e2e-api-a-dac2f3-serving-cert" not found
Jul 09 04:34:17.529578 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:17.529498    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls podName:07d8f3a4-24ba-41d0-a179-9dbec7f1c020 nodeName:}" failed. No retries permitted until 2026-07-09 04:34:18.029479445 +0000 UTC m=+2057.303857344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls") pod "maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" (UID: "07d8f3a4-24ba-41d0-a179-9dbec7f1c020") : secret "maas-api-e2e-api-a-dac2f3-serving-cert" not found
Jul 09 04:34:17.529961 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.529935    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.539229 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.539196    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s2pz\" (UniqueName: \"kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:17.594763 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.594687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:17.694629 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.694590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:17.794508 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.794448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:17.899925 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:17.899896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:18.034706 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.034653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:18.037302 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.037273    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") pod \"maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:18.210524 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.210422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:18.359807 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.359783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:18.361685 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:34:18.361662    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d8f3a4_24ba_41d0_a179_9dbec7f1c020.slice/crio-da01a77ee0713bc42de042d0912e235a211242fdfad0c12a2c8a1a64db61b426 WatchSource:0}: Error finding container da01a77ee0713bc42de042d0912e235a211242fdfad0c12a2c8a1a64db61b426: Status 404 returned error can't find the container with id da01a77ee0713bc42de042d0912e235a211242fdfad0c12a2c8a1a64db61b426
Jul 09 04:34:18.414988 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.414957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" event={"ID":"07d8f3a4-24ba-41d0-a179-9dbec7f1c020","Type":"ContainerStarted","Data":"da01a77ee0713bc42de042d0912e235a211242fdfad0c12a2c8a1a64db61b426"}
Jul 09 04:34:18.830175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.830150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:34:18.948215 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948230    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz664\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948407    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948437    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948513    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948568    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.948794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948594    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:18.948794 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.948606    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy\") pod \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\" (UID: \"9451c1d8-159a-43da-a6f9-1f6b1fe5e873\") "
Jul 09 04:34:18.949023 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.949000    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:18.949099 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.949013    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:34:18.949099 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.949042    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:18.949209 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.949183    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:18.950041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.950009    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data" (OuterVolumeSpecName: "istio-data") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:18.950995 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.950970    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664" (OuterVolumeSpecName: "kube-api-access-sz664") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "kube-api-access-sz664". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:18.950995 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.950986    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token" (OuterVolumeSpecName: "istio-token") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:18.951369 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.951347    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:18.951506 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:18.951377    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9451c1d8-159a-43da-a6f9-1f6b1fe5e873" (UID: "9451c1d8-159a-43da-a6f9-1f6b1fe5e873"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:34:19.049739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049708    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049731    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049739 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049742    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049750    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049759    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz664\" (UniqueName: \"kubernetes.io/projected/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-kube-api-access-sz664\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049768    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049777    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.049951 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.049793    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9451c1d8-159a-43da-a6f9-1f6b1fe5e873-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:19.421958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.421912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" event={"ID":"07d8f3a4-24ba-41d0-a179-9dbec7f1c020","Type":"ContainerStarted","Data":"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"}
Jul 09 04:34:19.422359 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.421982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:19.423639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.423616    2577 generic.go:356] "Generic (PLEG): container finished" podID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerID="e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15" exitCode=0
Jul 09 04:34:19.423769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.423654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerDied","Data":"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"}
Jul 09 04:34:19.423769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.423658    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"
Jul 09 04:34:19.423769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.423693    2577 scope.go:117] "RemoveContainer" containerID="e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"
Jul 09 04:34:19.423927 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.423685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh" event={"ID":"9451c1d8-159a-43da-a6f9-1f6b1fe5e873","Type":"ContainerDied","Data":"5e0dda27b35c500933b77591eeaefc738ce67723d2a8102838284a7deb63ef23"}
Jul 09 04:34:19.432714 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.432698    2577 scope.go:117] "RemoveContainer" containerID="b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"
Jul 09 04:34:19.438904 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.438839    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" podStartSLOduration=2.100014604 podStartE2EDuration="2.438826872s" totalImagesPullingTime="338.812268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:19.436590551 +0000 UTC m=+2058.710968459" watchObservedRunningTime="2026-07-09 04:34:19.438826872 +0000 UTC m=+2058.713204780"
Jul 09 04:34:19.442336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.442314    2577 scope.go:117] "RemoveContainer" containerID="e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"
Jul 09 04:34:19.442599 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:19.442577    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15\": container with ID starting with e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15 not found: ID does not exist" containerID="e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"
Jul 09 04:34:19.442709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.442608    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15"} err="failed to get container status \"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15\": rpc error: code = NotFound desc = could not find container \"e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15\": container with ID starting with e068a89536b08634e6950c271fdb7d791b2946ae6da9e0a0e04d5abe01987f15 not found: ID does not exist"
Jul 09 04:34:19.442709 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.442633    2577 scope.go:117] "RemoveContainer" containerID="b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"
Jul 09 04:34:19.442912 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:19.442894    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4\": container with ID starting with b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4 not found: ID does not exist" containerID="b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"
Jul 09 04:34:19.442958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.442918    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4"} err="failed to get container status \"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4\": rpc error: code = NotFound desc = could not find container \"b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4\": container with ID starting with b9cd8e54d89f479ae31f0ac17ebd81356e6cda78ef936c1ffbb78d0e3e370cd4 not found: ID does not exist"
Jul 09 04:34:19.450272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.450239    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:34:19.453708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:19.453689    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b3e4d0-openshift-default-5856b4f86c-qlrrh"]
Jul 09 04:34:21.180216 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:21.180183    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" path="/var/lib/kubelet/pods/9451c1d8-159a-43da-a6f9-1f6b1fe5e873/volumes"
Jul 09 04:34:21.687437 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:21.687389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:22.137093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.137053    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:22.435584 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.435483    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:22.436167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436146    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.436364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436169    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.436364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436183    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.436364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.436364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436275    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.436364 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.436292    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9451c1d8-159a-43da-a6f9-1f6b1fe5e873" containerName="istio-proxy"
Jul 09 04:34:22.441619 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.441595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:22.441758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.441699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.443721 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.443700    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1c4442-openshift-default-dockercfg-th4xt\""
Jul 09 04:34:22.453073 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.453021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:22.582344 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582530 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.582752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.582642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdlcz\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.683712 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.683867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.683867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdlcz\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.683867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.683867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.683980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684097 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684086    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684370 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684134    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684370 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684529 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684506    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.684657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.684640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.686156 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.686103    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.686305 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.686285    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.691240 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.691216    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.691418 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.691400    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdlcz\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz\") pod \"e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") " pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.753663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.753636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:22.840317 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.840284    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:22.903562 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.901166    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:22.903562 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:34:22.903462    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1bd772_e7a8_4b0e_a1a0_e7def3298fbf.slice/crio-14cfa47c5cb9128f384179b6657d8590af5a84acf7f2de19ff086c6a505673e9 WatchSource:0}: Error finding container 14cfa47c5cb9128f384179b6657d8590af5a84acf7f2de19ff086c6a505673e9: Status 404 returned error can't find the container with id 14cfa47c5cb9128f384179b6657d8590af5a84acf7f2de19ff086c6a505673e9
Jul 09 04:34:22.906630 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.906591    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:22.906752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.906674    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:22.906752 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:22.906721    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:23.451192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:23.451152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerStarted","Data":"414d1cad261ae326a6c2bf42e99adbc4d51de593ea34b0a96b8189ef47cdaa00"}
Jul 09 04:34:23.451692 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:23.451215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerStarted","Data":"14cfa47c5cb9128f384179b6657d8590af5a84acf7f2de19ff086c6a505673e9"}
Jul 09 04:34:23.469197 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:23.469146    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podStartSLOduration=1.469129015 podStartE2EDuration="1.469129015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:23.468118401 +0000 UTC m=+2062.742496351" watchObservedRunningTime="2026-07-09 04:34:23.469129015 +0000 UTC m=+2062.743506923"
Jul 09 04:34:23.754139 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:23.754059    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:23.759195 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:23.759174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:24.457202 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:24.457176    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:24.458037 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:24.458021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:25.388497 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.388434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:25.427743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.427711    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:25.431616 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.431595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.434160 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.433958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1c4442-serving-cert\""
Jul 09 04:34:25.439552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.439529    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:25.440514 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.440490    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:25.473675 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.473633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:25.532843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.532812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:25.611848 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.611815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.612013 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.611863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.612142 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.612116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgf62\" (UniqueName: \"kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.712941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.712862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgf62\" (UniqueName: \"kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.712941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.712917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.712941 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.712935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.713180 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:25.713022    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-serving-cert: secret "maas-api-e2e-api-b-1c4442-serving-cert" not found
Jul 09 04:34:25.713180 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:25.713080    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls podName:e5b7c987-e8ef-4f64-9c65-13e619ea1f0f nodeName:}" failed. No retries permitted until 2026-07-09 04:34:26.213065124 +0000 UTC m=+2065.487443010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls") pod "maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" (UID: "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f") : secret "maas-api-e2e-api-b-1c4442-serving-cert" not found
Jul 09 04:34:25.713574 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.713556    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.721259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.721242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgf62\" (UniqueName: \"kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:25.834565 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:25.834537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:26.008170 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.008061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:26.109753 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.109724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:26.219450 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.219412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:26.221938 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.221910    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") pod \"maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:26.347028 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.347002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:26.508930 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:26.505958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:26.511211 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:34:26.511182    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b7c987_e8ef_4f64_9c65_13e619ea1f0f.slice/crio-9b75fb8eb48e4b38b2c01348f67f1cc4ca435af94eaed9372b2eef877a4a8b50 WatchSource:0}: Error finding container 9b75fb8eb48e4b38b2c01348f67f1cc4ca435af94eaed9372b2eef877a4a8b50: Status 404 returned error can't find the container with id 9b75fb8eb48e4b38b2c01348f67f1cc4ca435af94eaed9372b2eef877a4a8b50
Jul 09 04:34:27.478159 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:27.478117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" event={"ID":"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f","Type":"ContainerStarted","Data":"3597df0b6e3365553f6fdc59c0f2a5108b82783ddf06d1eddd4659e3c9f7f347"}
Jul 09 04:34:27.478360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:27.478186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" event={"ID":"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f","Type":"ContainerStarted","Data":"9b75fb8eb48e4b38b2c01348f67f1cc4ca435af94eaed9372b2eef877a4a8b50"}
Jul 09 04:34:27.478360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:27.478218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:27.495490 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:27.494692    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" podStartSLOduration=1.9160322010000002 podStartE2EDuration="2.494661081s" totalImagesPullingTime="578.62888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:27.492379883 +0000 UTC m=+2066.766757790" watchObservedRunningTime="2026-07-09 04:34:27.494661081 +0000 UTC m=+2066.769038990"
Jul 09 04:34:28.596827 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:28.596790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:29.852907 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:29.852879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:33.489740 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:33.489710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:37.931986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:37.931912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:38.338091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.338014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:38.440320 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.440285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:38.537760 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.537728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:38.638869 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.638829    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:38.639194 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.639078    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" podUID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" containerName="maas-api" containerID="cri-o://4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f" gracePeriod=30
Jul 09 04:34:38.659087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.659064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:38.902158 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.902094    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:38.937184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.937154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s2pz\" (UniqueName: \"kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz\") pod \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") "
Jul 09 04:34:38.937655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.937307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca\") pod \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") "
Jul 09 04:34:38.937655 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.937404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") pod \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\" (UID: \"07d8f3a4-24ba-41d0-a179-9dbec7f1c020\") "
Jul 09 04:34:38.937765 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.937687    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07d8f3a4-24ba-41d0-a179-9dbec7f1c020" (UID: "07d8f3a4-24ba-41d0-a179-9dbec7f1c020"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:34:38.937841 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.937813    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:38.939947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.939916    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz" (OuterVolumeSpecName: "kube-api-access-6s2pz") pod "07d8f3a4-24ba-41d0-a179-9dbec7f1c020" (UID: "07d8f3a4-24ba-41d0-a179-9dbec7f1c020"). InnerVolumeSpecName "kube-api-access-6s2pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:38.940146 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:38.939979    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07d8f3a4-24ba-41d0-a179-9dbec7f1c020" (UID: "07d8f3a4-24ba-41d0-a179-9dbec7f1c020"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:34:39.038670 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.038627    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:39.038670 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.038659    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s2pz\" (UniqueName: \"kubernetes.io/projected/07d8f3a4-24ba-41d0-a179-9dbec7f1c020-kube-api-access-6s2pz\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:39.550521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.550426    2577 generic.go:356] "Generic (PLEG): container finished" podID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" containerID="4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f" exitCode=0
Jul 09 04:34:39.550521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.550508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" event={"ID":"07d8f3a4-24ba-41d0-a179-9dbec7f1c020","Type":"ContainerDied","Data":"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"}
Jul 09 04:34:39.550730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.550566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v" event={"ID":"07d8f3a4-24ba-41d0-a179-9dbec7f1c020","Type":"ContainerDied","Data":"da01a77ee0713bc42de042d0912e235a211242fdfad0c12a2c8a1a64db61b426"}
Jul 09 04:34:39.550730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.550580    2577 scope.go:117] "RemoveContainer" containerID="4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"
Jul 09 04:34:39.550730 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.550515    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"
Jul 09 04:34:39.560509 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.560487    2577 scope.go:117] "RemoveContainer" containerID="4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"
Jul 09 04:34:39.560793 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:39.560773    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f\": container with ID starting with 4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f not found: ID does not exist" containerID="4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"
Jul 09 04:34:39.560842 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.560800    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f"} err="failed to get container status \"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f\": rpc error: code = NotFound desc = could not find container \"4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f\": container with ID starting with 4fbe920f611432b83676de62963aa308603d5fda8719499f4c86eada9896ba4f not found: ID does not exist"
Jul 09 04:34:39.565426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.565404    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:39.568724 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:39.568707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dac2f3-9d5d494c8-sk88v"]
Jul 09 04:34:40.499658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:40.499602    2577 patch_prober.go:28] interesting pod/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 09 04:34:40.500203 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:40.500163    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 09 04:34:41.183365 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.183329    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" path="/var/lib/kubelet/pods/07d8f3a4-24ba-41d0-a179-9dbec7f1c020/volumes"
Jul 09 04:34:41.565816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.565730    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerID="414d1cad261ae326a6c2bf42e99adbc4d51de593ea34b0a96b8189ef47cdaa00" exitCode=0
Jul 09 04:34:41.566162 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.565806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerDied","Data":"414d1cad261ae326a6c2bf42e99adbc4d51de593ea34b0a96b8189ef47cdaa00"}
Jul 09 04:34:41.566208 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.566199    2577 scope.go:117] "RemoveContainer" containerID="414d1cad261ae326a6c2bf42e99adbc4d51de593ea34b0a96b8189ef47cdaa00"
Jul 09 04:34:41.567189 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.567161    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:41.567259 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.567241    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:41.567303 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:41.567289    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:34:42.574078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:42.574047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerStarted","Data":"865af6f58652b76533d110f255a292e9deaf8b0d364ca077bc46cad6ce38d34c"}
Jul 09 04:34:42.754522 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:42.754430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:42.754781 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:42.754758    2577 patch_prober.go:28] interesting pod/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 09 04:34:42.754864 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:42.754812    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 09 04:34:43.754899 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:43.754858    2577 patch_prober.go:28] interesting pod/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 09 04:34:43.755294 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:43.754927    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 09 04:34:44.107167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.107132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:44.205584 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.205552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:44.370685 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.370591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:44.589809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.589776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:44.744418 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.744281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:44.754350 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.754315    2577 patch_prober.go:28] interesting pod/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 09 04:34:44.754532 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:44.754399    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 09 04:34:45.140854 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.140808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:45.242311 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.242270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:45.343700 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.343660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:45.474808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.474725    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:45.474996 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.474966    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" podUID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" containerName="maas-api" containerID="cri-o://3597df0b6e3365553f6fdc59c0f2a5108b82783ddf06d1eddd4659e3c9f7f347" gracePeriod=30
Jul 09 04:34:45.511829 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.511803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:45.593154 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.593123    2577 generic.go:356] "Generic (PLEG): container finished" podID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" containerID="3597df0b6e3365553f6fdc59c0f2a5108b82783ddf06d1eddd4659e3c9f7f347" exitCode=0
Jul 09 04:34:45.593337 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.593215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" event={"ID":"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f","Type":"ContainerDied","Data":"3597df0b6e3365553f6fdc59c0f2a5108b82783ddf06d1eddd4659e3c9f7f347"}
Jul 09 04:34:45.757898 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.757863    2577 patch_prober.go:28] interesting pod/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 09 04:34:45.758149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.758118    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 09 04:34:45.766769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.766746    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:45.904649 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.904621    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgf62\" (UniqueName: \"kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62\") pod \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") "
Jul 09 04:34:45.904822 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.904711    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca\") pod \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") "
Jul 09 04:34:45.904822 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.904737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") pod \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\" (UID: \"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f\") "
Jul 09 04:34:45.905056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.905033    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" (UID: "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:34:45.906729 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.906704    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" (UID: "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:34:45.906877 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:45.906862    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62" (OuterVolumeSpecName: "kube-api-access-qgf62") pod "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" (UID: "e5b7c987-e8ef-4f64-9c65-13e619ea1f0f"). InnerVolumeSpecName "kube-api-access-qgf62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:46.005603 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.005519    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:46.005603 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.005550    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:46.005603 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.005559    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgf62\" (UniqueName: \"kubernetes.io/projected/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f-kube-api-access-qgf62\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:46.600583 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.600553    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"
Jul 09 04:34:46.600984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.600603    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c" event={"ID":"e5b7c987-e8ef-4f64-9c65-13e619ea1f0f","Type":"ContainerDied","Data":"9b75fb8eb48e4b38b2c01348f67f1cc4ca435af94eaed9372b2eef877a4a8b50"}
Jul 09 04:34:46.600984 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.600660    2577 scope.go:117] "RemoveContainer" containerID="3597df0b6e3365553f6fdc59c0f2a5108b82783ddf06d1eddd4659e3c9f7f347"
Jul 09 04:34:46.620750 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.620727    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:46.623060 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.623042    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c4442-6777cfc8c4-sbv5c"]
Jul 09 04:34:46.757428 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.757399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:46.757772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.757758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:46.758435 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:46.758416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:47.179981 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:47.179955    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" path="/var/lib/kubelet/pods/e5b7c987-e8ef-4f64-9c65-13e619ea1f0f/volumes"
Jul 09 04:34:51.182481 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.182448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:51.252558 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.252526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:51.418019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.417987    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:51.418205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.418182    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy" containerID="cri-o://865af6f58652b76533d110f255a292e9deaf8b0d364ca077bc46cad6ce38d34c" gracePeriod=30
Jul 09 04:34:51.437087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.437017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:51.684934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:51.684895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:52.037090 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:52.037060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:52.349026 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:52.349000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:52.450252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:52.450223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:55.349005 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.348973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:55.422897 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.422867    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:34:55.423293 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423281    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" containerName="maas-api"
Jul 09 04:34:55.423351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423294    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" containerName="maas-api"
Jul 09 04:34:55.423351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423319    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" containerName="maas-api"
Jul 09 04:34:55.423351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423324    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" containerName="maas-api"
Jul 09 04:34:55.423491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423387    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5b7c987-e8ef-4f64-9c65-13e619ea1f0f" containerName="maas-api"
Jul 09 04:34:55.423491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.423400    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d8f3a4-24ba-41d0-a179-9dbec7f1c020" containerName="maas-api"
Jul 09 04:34:55.428111 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.428089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.430039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.430019    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7b94ae-serving-cert\""
Jul 09 04:34:55.436444 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.436425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:34:55.440480 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.440443    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:55.491245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.491220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szv52\" (UniqueName: \"kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.491362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.491251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.491362 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.491292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.503654 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.503630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:55.592120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.592095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szv52\" (UniqueName: \"kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.592269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.592124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.592269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.592161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.592358 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:55.592284    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-serving-cert: secret "maas-api-e2e-shared-a-7b94ae-serving-cert" not found
Jul 09 04:34:55.592358 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:34:55.592346    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls podName:f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859 nodeName:}" failed. No retries permitted until 2026-07-09 04:34:56.092327392 +0000 UTC m=+2095.366705285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls") pod "maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" (UID: "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859") : secret "maas-api-e2e-shared-a-7b94ae-serving-cert" not found
Jul 09 04:34:55.592742 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.592722    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.599767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.599710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szv52\" (UniqueName: \"kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:55.834978 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:55.834939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:56.048218 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.048134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:56.096994 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.096958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:56.099483 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.099443    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:56.151776 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.151735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:34:56.342993 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.342897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:56.491044 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.491011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:34:56.504526 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:34:56.504497    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9327b3c_d3d1_4ac3_81f0_3b3f3e71b859.slice/crio-9131fe316254ef0f75181a49232a4947ba630a664a509267eb05d743ca78bf6b WatchSource:0}: Error finding container 9131fe316254ef0f75181a49232a4947ba630a664a509267eb05d743ca78bf6b: Status 404 returned error can't find the container with id 9131fe316254ef0f75181a49232a4947ba630a664a509267eb05d743ca78bf6b
Jul 09 04:34:56.662706 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.662670    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerID="865af6f58652b76533d110f255a292e9deaf8b0d364ca077bc46cad6ce38d34c" exitCode=0
Jul 09 04:34:56.662890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.662735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerDied","Data":"865af6f58652b76533d110f255a292e9deaf8b0d364ca077bc46cad6ce38d34c"}
Jul 09 04:34:56.662890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.662803    2577 scope.go:117] "RemoveContainer" containerID="414d1cad261ae326a6c2bf42e99adbc4d51de593ea34b0a96b8189ef47cdaa00"
Jul 09 04:34:56.664339 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.664316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" event={"ID":"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859","Type":"ContainerStarted","Data":"9131fe316254ef0f75181a49232a4947ba630a664a509267eb05d743ca78bf6b"}
Jul 09 04:34:56.701547 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.701525    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:56.804671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804640    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.804860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.804860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.804860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.804860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804809    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.804860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdlcz\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.805112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804897    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.805112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.804925    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.805112 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo\") pod \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\" (UID: \"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf\") "
Jul 09 04:34:56.805265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805145    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:34:56.805265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805149    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:56.805265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805166    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:56.805265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805251    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:56.805528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805400    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.805528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805416    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.805528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805427    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.805528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805441    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.805777 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.805758    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:56.807237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.807208    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:56.807237 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.807227    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz" (OuterVolumeSpecName: "kube-api-access-kdlcz") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "kube-api-access-kdlcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:34:56.807448 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.807431    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:34:56.807538 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.807438    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" (UID: "3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:34:56.906489 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.906450    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.906657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.906504    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.906657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.906514    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.906657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.906522    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:56.906657 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:56.906531    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdlcz\" (UniqueName: \"kubernetes.io/projected/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf-kube-api-access-kdlcz\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:34:57.672395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.672366    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"
Jul 09 04:34:57.672870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.672401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f" event={"ID":"3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf","Type":"ContainerDied","Data":"14cfa47c5cb9128f384179b6657d8590af5a84acf7f2de19ff086c6a505673e9"}
Jul 09 04:34:57.672870 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.672455    2577 scope.go:117] "RemoveContainer" containerID="865af6f58652b76533d110f255a292e9deaf8b0d364ca077bc46cad6ce38d34c"
Jul 09 04:34:57.674109 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.674075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" event={"ID":"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859","Type":"ContainerStarted","Data":"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"}
Jul 09 04:34:57.674252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.674233    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:34:57.693792 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.693754    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" podStartSLOduration=2.076984775 podStartE2EDuration="2.693742747s" totalImagesPullingTime="616.757972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:57.690315209 +0000 UTC m=+2096.964693116" watchObservedRunningTime="2026-07-09 04:34:57.693742747 +0000 UTC m=+2096.968120654"
Jul 09 04:34:57.704548 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.704523    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:57.711175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:57.711151    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1c4442-openshift-default-5c96ddbb5c-5876f"]
Jul 09 04:34:59.180520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:34:59.180463    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" path="/var/lib/kubelet/pods/3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf/volumes"
Jul 09 04:35:00.115266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.115238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:00.216588 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.216553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:00.433576 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.433498    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:35:00.434165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434146    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.434213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434169    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.434213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434185    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.434213 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434194    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.434314 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434308    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.434351 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.434323    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e1bd772-e7a8-4b0e-a1a0-e7def3298fbf" containerName="istio-proxy"
Jul 09 04:35:00.439160 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.439142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.440992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.440972    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-65509e-openshift-default-dockercfg-xqhw6\""
Jul 09 04:35:00.445816 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.445792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:35:00.479671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.479642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:00.538915 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.538881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.538974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539063    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52v4w\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.539270 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.539259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640105 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640321 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640660 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v4w\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640658    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640856 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640832    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.640913 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.640883    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.641135 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.641116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.642785 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.642760    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.642952 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.642937    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.647233 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.647202    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.647336 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.647278    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52v4w\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w\") pod \"e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") " pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.751252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.751177    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:00.767153 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.767124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:00.890445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.890421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:35:00.891937 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:35:00.891907    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a4613a_3cba_4352_9571_aa8f6e73c80a.slice/crio-8c2dfdc8902649909d9da58bb2f95667791ba2b7991415a339c2d0e68ac756b1 WatchSource:0}: Error finding container 8c2dfdc8902649909d9da58bb2f95667791ba2b7991415a339c2d0e68ac756b1: Status 404 returned error can't find the container with id 8c2dfdc8902649909d9da58bb2f95667791ba2b7991415a339c2d0e68ac756b1
Jul 09 04:35:00.894187 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.894154    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:00.894272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.894218    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:00.894272 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:00.894245    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:01.282887 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.282863    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:35:01.293121 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.293100    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:35:01.701031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.700994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerStarted","Data":"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"}
Jul 09 04:35:01.701203 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.701087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerStarted","Data":"8c2dfdc8902649909d9da58bb2f95667791ba2b7991415a339c2d0e68ac756b1"}
Jul 09 04:35:01.719281 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.719236    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podStartSLOduration=1.719222231 podStartE2EDuration="1.719222231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:35:01.717088165 +0000 UTC m=+2100.991466073" watchObservedRunningTime="2026-07-09 04:35:01.719222231 +0000 UTC m=+2100.993600138"
Jul 09 04:35:01.751363 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.751330    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:01.757327 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:01.757302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:02.707113 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:02.707076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:02.708053 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:02.708033    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:03.609120 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.609087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:03.665377 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.665343    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:35:03.671193 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.671165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.673196 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.673166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-65509e-serving-cert\""
Jul 09 04:35:03.680879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.680855    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:35:03.685266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.685244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:03.686723 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.686706    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:35:03.765722 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.765674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:03.768863 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.768841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4snzx\" (UniqueName: \"kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.768934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.768889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.768972 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.768945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.869798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.869705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4snzx\" (UniqueName: \"kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.870161 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.870136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.870303 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.870203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.870376 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:35:03.870319    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-serving-cert: secret "maas-api-e2e-shared-b-65509e-serving-cert" not found
Jul 09 04:35:03.870434 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:35:03.870391    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls podName:47ca528b-83ae-495c-91de-34cb4505f361 nodeName:}" failed. No retries permitted until 2026-07-09 04:35:04.370369824 +0000 UTC m=+2103.644747714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls") pod "maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" (UID: "47ca528b-83ae-495c-91de-34cb4505f361") : secret "maas-api-e2e-shared-b-65509e-serving-cert" not found
Jul 09 04:35:03.870727 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.870709    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:03.879389 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:03.879363    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4snzx\" (UniqueName: \"kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:04.258953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:04.258865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:04.374798 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:04.374758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:04.374963 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:35:04.374910    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-serving-cert: secret "maas-api-e2e-shared-b-65509e-serving-cert" not found
Jul 09 04:35:04.375027 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:35:04.374972    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls podName:47ca528b-83ae-495c-91de-34cb4505f361 nodeName:}" failed. No retries permitted until 2026-07-09 04:35:05.374957143 +0000 UTC m=+2104.649335030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls") pod "maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" (UID: "47ca528b-83ae-495c-91de-34cb4505f361") : secret "maas-api-e2e-shared-b-65509e-serving-cert" not found
Jul 09 04:35:04.407860 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:04.407818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:05.033810 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.033780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:05.385445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.385415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:05.387753 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.387728    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:05.485142 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.485112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:05.636208 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.636184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:35:05.638340 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:35:05.638310    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ca528b_83ae_495c_91de_34cb4505f361.slice/crio-0728e83b4bd52968fc4affa7ff0b8cc64c3ac2c9beff602f97131d22ef4cbf16 WatchSource:0}: Error finding container 0728e83b4bd52968fc4affa7ff0b8cc64c3ac2c9beff602f97131d22ef4cbf16: Status 404 returned error can't find the container with id 0728e83b4bd52968fc4affa7ff0b8cc64c3ac2c9beff602f97131d22ef4cbf16
Jul 09 04:35:05.727019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:05.726981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" event={"ID":"47ca528b-83ae-495c-91de-34cb4505f361","Type":"ContainerStarted","Data":"0728e83b4bd52968fc4affa7ff0b8cc64c3ac2c9beff602f97131d22ef4cbf16"}
Jul 09 04:35:06.736998 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:06.736951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" event={"ID":"47ca528b-83ae-495c-91de-34cb4505f361","Type":"ContainerStarted","Data":"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"}
Jul 09 04:35:06.737511 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:06.737053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:06.761934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:06.761877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" podStartSLOduration=3.391336686 podStartE2EDuration="3.761864121s" totalImagesPullingTime="370.527435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:35:06.758530245 +0000 UTC m=+2106.032908150" watchObservedRunningTime="2026-07-09 04:35:06.761864121 +0000 UTC m=+2106.036242028"
Jul 09 04:35:08.087404 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:08.087372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:12.754932 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:12.754903    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:35:17.682509 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.682460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:17.780079 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.779005    2577 patch_prober.go:28] interesting pod/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": read tcp 10.134.0.2:54688->10.134.0.80:15021: read: connection reset by peer" start-of-body=
Jul 09 04:35:17.780079 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.779069    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": read tcp 10.134.0.2:54688->10.134.0.80:15021: read: connection reset by peer"
Jul 09 04:35:17.782937 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.782901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:17.805335 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.805310    2577 generic.go:356] "Generic (PLEG): container finished" podID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerID="46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3" exitCode=0
Jul 09 04:35:17.805445 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.805380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerDied","Data":"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"}
Jul 09 04:35:17.805841 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.805827    2577 scope.go:117] "RemoveContainer" containerID="46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"
Jul 09 04:35:17.806843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.806815    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:17.806903 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.806890    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:17.806940 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:17.806920    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 04:35:18.814310 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:18.814273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerStarted","Data":"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"}
Jul 09 04:35:19.752283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:19.752238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:19.752527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:19.752385    2577 patch_prober.go:28] interesting pod/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Jul 09 04:35:19.752527 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:19.752432    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Jul 09 04:35:19.945601 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:19.945553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:20.050678 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:20.050569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:20.752705 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:20.752604    2577 patch_prober.go:28] interesting pod/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Jul 09 04:35:20.752705 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:20.752688    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Jul 09 04:35:21.751761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:21.751720    2577 patch_prober.go:28] interesting pod/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Jul 09 04:35:21.752223 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:21.751784    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Jul 09 04:35:22.551494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:22.550919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:22.755314 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:22.755281    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:22.755767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:22.755575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:22.756235 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:22.756221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:35:25.348075 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:25.348040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:25.449874 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:25.449843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:35:37.708179 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:35:37.708145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:36:37.139261 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:36:37.139226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:36:52.660599 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:36:52.660569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:36:52.715725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:36:52.715696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:37:48.209770 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:37:48.209700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:37:49.482224 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:37:49.482193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:05.125608 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:05.125579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:16.797791 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:16.797718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:33.369934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:33.369899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:33.733746 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:33.733660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:42.451634 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:42.451604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:38:55.332244 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:38:55.332161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:39:14.738637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:39:14.738595    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:39:15.715767 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:39:15.715739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:39:26.002568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:39:26.002532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:01.321420 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:01.321394    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:40:01.335389 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:01.335362    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:40:17.812078 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:17.811977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:18.729197 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:18.729163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:18.831029 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:18.830992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:20.750337 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:20.750286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:20.854252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:20.854217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:26.571831 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:26.571749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:40:26.635217 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:40:26.635184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:15.441567 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:15.441528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:16.411972 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:16.411935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:17.320772 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.320738    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7530-140361435842224"]
Jul 09 04:41:17.324433 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.324417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:17.326205 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.326187    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-d5mg6\""
Jul 09 04:41:17.330484 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.330446    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361435842224"]
Jul 09 04:41:17.366981 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.366955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wrp\" (UniqueName: \"kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp\") pod \"test-curl-7530-140361435842224\" (UID: \"2b35965c-6463-4f5a-a662-b87759ab3305\") " pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:17.467843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.467811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wrp\" (UniqueName: \"kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp\") pod \"test-curl-7530-140361435842224\" (UID: \"2b35965c-6463-4f5a-a662-b87759ab3305\") " pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:17.475765 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.475737    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wrp\" (UniqueName: \"kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp\") pod \"test-curl-7530-140361435842224\" (UID: \"2b35965c-6463-4f5a-a662-b87759ab3305\") " pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:17.635317 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.635282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:17.775658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.775627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361435842224"]
Jul 09 04:41:17.777801 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:41:17.777770    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b35965c_6463_4f5a_a662_b87759ab3305.slice/crio-f4ad76cc86e486887582d499c69ca00da62fdf203813bb83b06a11e81049df90 WatchSource:0}: Error finding container f4ad76cc86e486887582d499c69ca00da62fdf203813bb83b06a11e81049df90: Status 404 returned error can't find the container with id f4ad76cc86e486887582d499c69ca00da62fdf203813bb83b06a11e81049df90
Jul 09 04:41:17.779149 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.779131    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:41:17.882538 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:17.882497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361435842224" event={"ID":"2b35965c-6463-4f5a-a662-b87759ab3305","Type":"ContainerStarted","Data":"f4ad76cc86e486887582d499c69ca00da62fdf203813bb83b06a11e81049df90"}
Jul 09 04:41:18.889572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:18.889537    2577 generic.go:356] "Generic (PLEG): container finished" podID="2b35965c-6463-4f5a-a662-b87759ab3305" containerID="a5347b106d6b063393b3f11312416fa8cd88ec5b4ca92fc11d655579541fde8d" exitCode=0
Jul 09 04:41:18.889956 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:18.889626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361435842224" event={"ID":"2b35965c-6463-4f5a-a662-b87759ab3305","Type":"ContainerDied","Data":"a5347b106d6b063393b3f11312416fa8cd88ec5b4ca92fc11d655579541fde8d"}
Jul 09 04:41:20.047885 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.047860    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:20.065384 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.065355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7530-140361435842224_2b35965c-6463-4f5a-a662-b87759ab3305/test-curl-7530-140361435842224/0.log"
Jul 09 04:41:20.086187 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.086157    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7530-140361435842224"]
Jul 09 04:41:20.089169 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.089143    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7530-140361435842224"]
Jul 09 04:41:20.094426 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.094405    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9wrp\" (UniqueName: \"kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp\") pod \"2b35965c-6463-4f5a-a662-b87759ab3305\" (UID: \"2b35965c-6463-4f5a-a662-b87759ab3305\") "
Jul 09 04:41:20.096415 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.096392    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp" (OuterVolumeSpecName: "kube-api-access-v9wrp") pod "2b35965c-6463-4f5a-a662-b87759ab3305" (UID: "2b35965c-6463-4f5a-a662-b87759ab3305"). InnerVolumeSpecName "kube-api-access-v9wrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:20.195980 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.195898    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9wrp\" (UniqueName: \"kubernetes.io/projected/2b35965c-6463-4f5a-a662-b87759ab3305-kube-api-access-v9wrp\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:20.205421 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.205387    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7530-140361435843632"]
Jul 09 04:41:20.205947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.205928    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" containerName="test-curl-7530-140361435842224"
Jul 09 04:41:20.206036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.205949    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" containerName="test-curl-7530-140361435842224"
Jul 09 04:41:20.206093 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.206052    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" containerName="test-curl-7530-140361435842224"
Jul 09 04:41:20.209281 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.209263    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:20.214347 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.214326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361435843632"]
Jul 09 04:41:20.296562 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.296533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmt77\" (UniqueName: \"kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77\") pod \"test-curl-7530-140361435843632\" (UID: \"fce03605-53da-43cf-95f1-137d105e8b04\") " pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:20.397308 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.397273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmt77\" (UniqueName: \"kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77\") pod \"test-curl-7530-140361435843632\" (UID: \"fce03605-53da-43cf-95f1-137d105e8b04\") " pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:20.405188 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.405164    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmt77\" (UniqueName: \"kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77\") pod \"test-curl-7530-140361435843632\" (UID: \"fce03605-53da-43cf-95f1-137d105e8b04\") " pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:20.519683 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.519604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:20.652671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.652647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361435843632"]
Jul 09 04:41:20.654819 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:41:20.654797    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfce03605_53da_43cf_95f1_137d105e8b04.slice/crio-1d7a9b1cb1891126d71a53cf980d8886b4058a3b12efe51d325cb2b1acee421a WatchSource:0}: Error finding container 1d7a9b1cb1891126d71a53cf980d8886b4058a3b12efe51d325cb2b1acee421a: Status 404 returned error can't find the container with id 1d7a9b1cb1891126d71a53cf980d8886b4058a3b12efe51d325cb2b1acee421a
Jul 09 04:41:20.902119 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.902080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361435843632" event={"ID":"fce03605-53da-43cf-95f1-137d105e8b04","Type":"ContainerStarted","Data":"1d7a9b1cb1891126d71a53cf980d8886b4058a3b12efe51d325cb2b1acee421a"}
Jul 09 04:41:20.903287 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.903268    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:20.903385 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:20.903276    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4ad76cc86e486887582d499c69ca00da62fdf203813bb83b06a11e81049df90"
Jul 09 04:41:20.913410 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:20.913387    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7530-140361435842224\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:21.178420 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:21.178326    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7530-140361435842224\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" pod="openshift-ingress/test-curl-7530-140361435842224"
Jul 09 04:41:21.180583 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:21.180557    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b35965c-6463-4f5a-a662-b87759ab3305" path="/var/lib/kubelet/pods/2b35965c-6463-4f5a-a662-b87759ab3305/volumes"
Jul 09 04:41:21.909814 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:21.909780    2577 generic.go:356] "Generic (PLEG): container finished" podID="fce03605-53da-43cf-95f1-137d105e8b04" containerID="c573bca7696a209260876073288fadab06e4091ccb24c5b5ff7dc771b72e14a3" exitCode=0
Jul 09 04:41:21.909986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:21.909857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361435843632" event={"ID":"fce03605-53da-43cf-95f1-137d105e8b04","Type":"ContainerDied","Data":"c573bca7696a209260876073288fadab06e4091ccb24c5b5ff7dc771b72e14a3"}
Jul 09 04:41:23.065425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.065403    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:23.084687 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.084665    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7530-140361435843632_fce03605-53da-43cf-95f1-137d105e8b04/test-curl-7530-140361435843632/0.log"
Jul 09 04:41:23.109411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.109383    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7530-140361435843632"]
Jul 09 04:41:23.113491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.113447    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7530-140361435843632"]
Jul 09 04:41:23.121441 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.121422    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmt77\" (UniqueName: \"kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77\") pod \"fce03605-53da-43cf-95f1-137d105e8b04\" (UID: \"fce03605-53da-43cf-95f1-137d105e8b04\") "
Jul 09 04:41:23.123391 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.123365    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77" (OuterVolumeSpecName: "kube-api-access-vmt77") pod "fce03605-53da-43cf-95f1-137d105e8b04" (UID: "fce03605-53da-43cf-95f1-137d105e8b04"). InnerVolumeSpecName "kube-api-access-vmt77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:23.180245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.180168    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fce03605-53da-43cf-95f1-137d105e8b04" path="/var/lib/kubelet/pods/fce03605-53da-43cf-95f1-137d105e8b04/volumes"
Jul 09 04:41:23.223041 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.222999    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmt77\" (UniqueName: \"kubernetes.io/projected/fce03605-53da-43cf-95f1-137d105e8b04-kube-api-access-vmt77\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:23.269958 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.269924    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:23.270379 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.270366    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce03605-53da-43cf-95f1-137d105e8b04" containerName="test-curl-7530-140361435843632"
Jul 09 04:41:23.270423 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.270380    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce03605-53da-43cf-95f1-137d105e8b04" containerName="test-curl-7530-140361435843632"
Jul 09 04:41:23.270458 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.270454    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fce03605-53da-43cf-95f1-137d105e8b04" containerName="test-curl-7530-140361435843632"
Jul 09 04:41:23.273761 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.273739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:23.277386 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.277360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:23.324186 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.324158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjt44\" (UniqueName: \"kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44\") pod \"test-curl-7530-140361441257344\" (UID: \"98120f77-23f4-478c-995b-96660f448eff\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:23.424915 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.424882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjt44\" (UniqueName: \"kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44\") pod \"test-curl-7530-140361441257344\" (UID: \"98120f77-23f4-478c-995b-96660f448eff\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:23.433279 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.433198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjt44\" (UniqueName: \"kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44\") pod \"test-curl-7530-140361441257344\" (UID: \"98120f77-23f4-478c-995b-96660f448eff\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:23.584018 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.583979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:23.721410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.721385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:23.722880 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:41:23.722855    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98120f77_23f4_478c_995b_96660f448eff.slice/crio-0c6dc39f19956920c3c12d635627fee2c263fbba073bae241e86d36b1f4e0674 WatchSource:0}: Error finding container 0c6dc39f19956920c3c12d635627fee2c263fbba073bae241e86d36b1f4e0674: Status 404 returned error can't find the container with id 0c6dc39f19956920c3c12d635627fee2c263fbba073bae241e86d36b1f4e0674
Jul 09 04:41:23.922221 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.922193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361435843632"
Jul 09 04:41:23.922221 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.922222    2577 scope.go:117] "RemoveContainer" containerID="c573bca7696a209260876073288fadab06e4091ccb24c5b5ff7dc771b72e14a3"
Jul 09 04:41:23.923581 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:23.923558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441257344" event={"ID":"98120f77-23f4-478c-995b-96660f448eff","Type":"ContainerStarted","Data":"0c6dc39f19956920c3c12d635627fee2c263fbba073bae241e86d36b1f4e0674"}
Jul 09 04:41:24.825625 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:24.825585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:24.878900 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:24.878851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:24.931590 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:24.931555    2577 generic.go:356] "Generic (PLEG): container finished" podID="98120f77-23f4-478c-995b-96660f448eff" containerID="94123fcfc4e2e2dbeef1543b0f6ef73ac9cd5a0def93828e4bf0a9c7c39dd642" exitCode=0
Jul 09 04:41:24.931743 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:24.931648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441257344" event={"ID":"98120f77-23f4-478c-995b-96660f448eff","Type":"ContainerDied","Data":"94123fcfc4e2e2dbeef1543b0f6ef73ac9cd5a0def93828e4bf0a9c7c39dd642"}
Jul 09 04:41:26.083663 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.083640    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.102774 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.102743    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7530-140361441257344_98120f77-23f4-478c-995b-96660f448eff/test-curl-7530-140361441257344/0.log"
Jul 09 04:41:26.126111 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.126076    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:26.131845 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.131815    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:26.152201 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.152170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjt44\" (UniqueName: \"kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44\") pod \"98120f77-23f4-478c-995b-96660f448eff\" (UID: \"98120f77-23f4-478c-995b-96660f448eff\") "
Jul 09 04:41:26.154370 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.154340    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44" (OuterVolumeSpecName: "kube-api-access-jjt44") pod "98120f77-23f4-478c-995b-96660f448eff" (UID: "98120f77-23f4-478c-995b-96660f448eff"). InnerVolumeSpecName "kube-api-access-jjt44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:26.248345 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.248277    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:26.248751 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.248738    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98120f77-23f4-478c-995b-96660f448eff" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:26.248808 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.248753    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98120f77-23f4-478c-995b-96660f448eff" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:26.248845 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.248818    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98120f77-23f4-478c-995b-96660f448eff" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:26.252595 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.252018    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.253266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.253243    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjt44\" (UniqueName: \"kubernetes.io/projected/98120f77-23f4-478c-995b-96660f448eff-kube-api-access-jjt44\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:26.254966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.254945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:26.353691 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.353658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rs9n\" (UniqueName: \"kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n\") pod \"test-curl-7530-140361441257344\" (UID: \"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.455069 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.455034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rs9n\" (UniqueName: \"kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n\") pod \"test-curl-7530-140361441257344\" (UID: \"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.462307 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.462276    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rs9n\" (UniqueName: \"kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n\") pod \"test-curl-7530-140361441257344\" (UID: \"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8\") " pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.561589 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.561522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.696573 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.696544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:26.698136 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:41:26.698104    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee714fc8_c7ef_4a9d_8ca4_c6b0d37f96f8.slice/crio-91dbede032dbe848840956ee1817f0e9d0c44c330dddee32029e8a07da36017f WatchSource:0}: Error finding container 91dbede032dbe848840956ee1817f0e9d0c44c330dddee32029e8a07da36017f: Status 404 returned error can't find the container with id 91dbede032dbe848840956ee1817f0e9d0c44c330dddee32029e8a07da36017f
Jul 09 04:41:26.945256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.945227    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c6dc39f19956920c3c12d635627fee2c263fbba073bae241e86d36b1f4e0674"
Jul 09 04:41:26.945256 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.945227    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:26.946534 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.946503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441257344" event={"ID":"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8","Type":"ContainerStarted","Data":"91dbede032dbe848840956ee1817f0e9d0c44c330dddee32029e8a07da36017f"}
Jul 09 04:41:26.957820 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:26.957789    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7530-140361441257344" oldPodUID="98120f77-23f4-478c-995b-96660f448eff" podUID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8"
Jul 09 04:41:27.180844 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:27.180818    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98120f77-23f4-478c-995b-96660f448eff" path="/var/lib/kubelet/pods/98120f77-23f4-478c-995b-96660f448eff/volumes"
Jul 09 04:41:27.954138 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:27.954106    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" containerID="24ac275c94625db313a3aefe4c4a9342771ab25285b04104dd0236adbdb624b7" exitCode=0
Jul 09 04:41:27.954323 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:27.954194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441257344" event={"ID":"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8","Type":"ContainerDied","Data":"24ac275c94625db313a3aefe4c4a9342771ab25285b04104dd0236adbdb624b7"}
Jul 09 04:41:29.111176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.111148    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:29.130056 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.130030    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7530-140361441257344_ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8/test-curl-7530-140361441257344/0.log"
Jul 09 04:41:29.156022 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.155995    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:29.160297 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.160270    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7530-140361441257344"]
Jul 09 04:41:29.180293 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.180272    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rs9n\" (UniqueName: \"kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n\") pod \"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8\" (UID: \"ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8\") "
Jul 09 04:41:29.182279 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.182251    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n" (OuterVolumeSpecName: "kube-api-access-5rs9n") pod "ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" (UID: "ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8"). InnerVolumeSpecName "kube-api-access-5rs9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:29.270378 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.270311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7530-140361441258352"]
Jul 09 04:41:29.270773 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.270760    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:29.270817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.270775    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:29.270857 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.270851    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" containerName="test-curl-7530-140361441257344"
Jul 09 04:41:29.273934 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.273920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:29.280985 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.280784    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441258352"]
Jul 09 04:41:29.281143 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.281114    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rs9n\" (UniqueName: \"kubernetes.io/projected/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8-kube-api-access-5rs9n\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:29.381964 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.381923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbzrp\" (UniqueName: \"kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp\") pod \"test-curl-7530-140361441258352\" (UID: \"9504263d-d346-4a80-aa63-34af7098c101\") " pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:29.483516 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.483453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbzrp\" (UniqueName: \"kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp\") pod \"test-curl-7530-140361441258352\" (UID: \"9504263d-d346-4a80-aa63-34af7098c101\") " pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:29.491269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.491241    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbzrp\" (UniqueName: \"kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp\") pod \"test-curl-7530-140361441258352\" (UID: \"9504263d-d346-4a80-aa63-34af7098c101\") " pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:29.584255 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.584170    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:29.723274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.723241    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7530-140361441258352"]
Jul 09 04:41:29.724886 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:41:29.724863    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9504263d_d346_4a80_aa63_34af7098c101.slice/crio-0c9536e06257b529af24c2ac477c2e07d8dcc0ed6f7ed5bbe3f4a1b90dbdfa05 WatchSource:0}: Error finding container 0c9536e06257b529af24c2ac477c2e07d8dcc0ed6f7ed5bbe3f4a1b90dbdfa05: Status 404 returned error can't find the container with id 0c9536e06257b529af24c2ac477c2e07d8dcc0ed6f7ed5bbe3f4a1b90dbdfa05
Jul 09 04:41:29.966769 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.966735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441258352" event={"ID":"9504263d-d346-4a80-aa63-34af7098c101","Type":"ContainerStarted","Data":"0c9536e06257b529af24c2ac477c2e07d8dcc0ed6f7ed5bbe3f4a1b90dbdfa05"}
Jul 09 04:41:29.968124 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.968108    2577 scope.go:117] "RemoveContainer" containerID="24ac275c94625db313a3aefe4c4a9342771ab25285b04104dd0236adbdb624b7"
Jul 09 04:41:29.968247 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:29.968134    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441257344"
Jul 09 04:41:31.180556 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:31.180521    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8" path="/var/lib/kubelet/pods/ee714fc8-c7ef-4a9d-8ca4-c6b0d37f96f8/volumes"
Jul 09 04:41:31.981893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:31.981858    2577 generic.go:356] "Generic (PLEG): container finished" podID="9504263d-d346-4a80-aa63-34af7098c101" containerID="daff0acc92ec81481f4fbc8603753b71c12441cd38d9edc7652cad67c6833648" exitCode=0
Jul 09 04:41:31.982066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:31.981909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7530-140361441258352" event={"ID":"9504263d-d346-4a80-aa63-34af7098c101","Type":"ContainerDied","Data":"daff0acc92ec81481f4fbc8603753b71c12441cd38d9edc7652cad67c6833648"}
Jul 09 04:41:33.128720 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.128693    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:33.147930 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.147902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7530-140361441258352_9504263d-d346-4a80-aa63-34af7098c101/test-curl-7530-140361441258352/0.log"
Jul 09 04:41:33.172572 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.172541    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7530-140361441258352"]
Jul 09 04:41:33.186265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.186238    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7530-140361441258352"]
Jul 09 04:41:33.218983 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.218957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbzrp\" (UniqueName: \"kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp\") pod \"9504263d-d346-4a80-aa63-34af7098c101\" (UID: \"9504263d-d346-4a80-aa63-34af7098c101\") "
Jul 09 04:41:33.220989 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.220956    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp" (OuterVolumeSpecName: "kube-api-access-qbzrp") pod "9504263d-d346-4a80-aa63-34af7098c101" (UID: "9504263d-d346-4a80-aa63-34af7098c101"). InnerVolumeSpecName "kube-api-access-qbzrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:33.320459 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.320392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbzrp\" (UniqueName: \"kubernetes.io/projected/9504263d-d346-4a80-aa63-34af7098c101-kube-api-access-qbzrp\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:33.993843 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.993817    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:33.994014 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:33.993852    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c9536e06257b529af24c2ac477c2e07d8dcc0ed6f7ed5bbe3f4a1b90dbdfa05"
Jul 09 04:41:34.002924 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:34.002895    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7530-140361441258352\" is forbidden: User \"system:node:ip-10-0-135-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-255.ec2.internal' and this object" podUID="9504263d-d346-4a80-aa63-34af7098c101" pod="openshift-ingress/test-curl-7530-140361441258352"
Jul 09 04:41:35.180520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:35.180457    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9504263d-d346-4a80-aa63-34af7098c101" path="/var/lib/kubelet/pods/9504263d-d346-4a80-aa63-34af7098c101/volumes"
Jul 09 04:41:36.853252 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:36.853214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:37.248541 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.248437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:37.347893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.347864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:37.448878 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.448846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:37.509245 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.509149    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:41:37.509620 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.509588    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" podUID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" containerName="maas-api" containerID="cri-o://b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1" gracePeriod=30
Jul 09 04:41:37.548569 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.548541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:37.972395 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:37.972371    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:41:38.019167 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.019137    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" containerID="b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1" exitCode=0
Jul 09 04:41:38.019324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.019179    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"
Jul 09 04:41:38.019324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.019229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" event={"ID":"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859","Type":"ContainerDied","Data":"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"}
Jul 09 04:41:38.019324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.019284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr" event={"ID":"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859","Type":"ContainerDied","Data":"9131fe316254ef0f75181a49232a4947ba630a664a509267eb05d743ca78bf6b"}
Jul 09 04:41:38.019324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.019312    2577 scope.go:117] "RemoveContainer" containerID="b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"
Jul 09 04:41:38.029528 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.029505    2577 scope.go:117] "RemoveContainer" containerID="b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"
Jul 09 04:41:38.029804 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:38.029784    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1\": container with ID starting with b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1 not found: ID does not exist" containerID="b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"
Jul 09 04:41:38.029912 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.029809    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1"} err="failed to get container status \"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1\": rpc error: code = NotFound desc = could not find container \"b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1\": container with ID starting with b9318ad1ae50f599a907a82fc923e3501a40e988c6affcebe84cd7d1f14011e1 not found: ID does not exist"
Jul 09 04:41:38.061687 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.061633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca\") pod \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") "
Jul 09 04:41:38.061779 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.061699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szv52\" (UniqueName: \"kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52\") pod \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") "
Jul 09 04:41:38.061779 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.061748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") pod \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\" (UID: \"f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859\") "
Jul 09 04:41:38.062039 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.062013    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" (UID: "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:41:38.063658 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.063634    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52" (OuterVolumeSpecName: "kube-api-access-szv52") pod "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" (UID: "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859"). InnerVolumeSpecName "kube-api-access-szv52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:38.063734 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.063637    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" (UID: "f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:41:38.163410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.163381    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:38.163410 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.163408    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:38.163590 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.163423    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szv52\" (UniqueName: \"kubernetes.io/projected/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859-kube-api-access-szv52\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:38.342520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.342484    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:41:38.345007 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:38.344988    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b94ae-5d79d7458-x44tr"]
Jul 09 04:41:39.179725 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:39.179697    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" path="/var/lib/kubelet/pods/f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859/volumes"
Jul 09 04:41:43.012130 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:43.012101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:43.102570 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:43.102519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:43.262608 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:43.262495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:43.474928 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:43.474901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:43.626651 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:43.626621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:44.021672 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.021589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:44.120817 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.120773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:44.221249 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.221218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:44.286873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.286793    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:41:44.287132 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.287105    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" podUID="47ca528b-83ae-495c-91de-34cb4505f361" containerName="maas-api" containerID="cri-o://1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea" gracePeriod=30
Jul 09 04:41:44.308424 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.308400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:44.544274 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.544216    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:41:44.627190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.627160    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca\") pod \"47ca528b-83ae-495c-91de-34cb4505f361\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") "
Jul 09 04:41:44.627190 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.627193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") pod \"47ca528b-83ae-495c-91de-34cb4505f361\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") "
Jul 09 04:41:44.627408 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.627254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4snzx\" (UniqueName: \"kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx\") pod \"47ca528b-83ae-495c-91de-34cb4505f361\" (UID: \"47ca528b-83ae-495c-91de-34cb4505f361\") "
Jul 09 04:41:44.627609 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.627585    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47ca528b-83ae-495c-91de-34cb4505f361" (UID: "47ca528b-83ae-495c-91de-34cb4505f361"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:41:44.629338 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.629316    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47ca528b-83ae-495c-91de-34cb4505f361" (UID: "47ca528b-83ae-495c-91de-34cb4505f361"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:41:44.629758 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.629741    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx" (OuterVolumeSpecName: "kube-api-access-4snzx") pod "47ca528b-83ae-495c-91de-34cb4505f361" (UID: "47ca528b-83ae-495c-91de-34cb4505f361"). InnerVolumeSpecName "kube-api-access-4snzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:44.728104 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.728066    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47ca528b-83ae-495c-91de-34cb4505f361-openshift-service-ca\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:44.728104 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.728099    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47ca528b-83ae-495c-91de-34cb4505f361-maas-api-tls\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:44.728301 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:44.728112    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4snzx\" (UniqueName: \"kubernetes.io/projected/47ca528b-83ae-495c-91de-34cb4505f361-kube-api-access-4snzx\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:45.063091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.063054    2577 generic.go:356] "Generic (PLEG): container finished" podID="47ca528b-83ae-495c-91de-34cb4505f361" containerID="1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea" exitCode=0
Jul 09 04:41:45.063504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.063103    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"
Jul 09 04:41:45.063504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.063145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" event={"ID":"47ca528b-83ae-495c-91de-34cb4505f361","Type":"ContainerDied","Data":"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"}
Jul 09 04:41:45.063504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.063192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7" event={"ID":"47ca528b-83ae-495c-91de-34cb4505f361","Type":"ContainerDied","Data":"0728e83b4bd52968fc4affa7ff0b8cc64c3ac2c9beff602f97131d22ef4cbf16"}
Jul 09 04:41:45.063504 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.063219    2577 scope.go:117] "RemoveContainer" containerID="1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"
Jul 09 04:41:45.073184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.073167    2577 scope.go:117] "RemoveContainer" containerID="1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"
Jul 09 04:41:45.073456 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:45.073434    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea\": container with ID starting with 1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea not found: ID does not exist" containerID="1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"
Jul 09 04:41:45.073525 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.073479    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea"} err="failed to get container status \"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea\": rpc error: code = NotFound desc = could not find container \"1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea\": container with ID starting with 1c27219b7c7e70d25277b0bf1cc3857acb566b19316d9add1dc493279a19b2ea not found: ID does not exist"
Jul 09 04:41:45.084719 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.084697    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:41:45.091271 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.091250    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-65509e-77d6c78df5-tr5c7"]
Jul 09 04:41:45.180926 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:45.180897    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ca528b-83ae-495c-91de-34cb4505f361" path="/var/lib/kubelet/pods/47ca528b-83ae-495c-91de-34cb4505f361/volumes"
Jul 09 04:41:49.913677 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:49.913642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:50.016708 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.016674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:50.134707 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.134670    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:41:50.134935 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.134910    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" containerID="cri-o://d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d" gracePeriod=30
Jul 09 04:41:50.148831 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.148809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:50.368491 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.368434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:41:50.763879 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:50.763797    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8869bc4d-jt4kx_38e18490-3acd-4e52-b2b6-ca7ac1b3faeb/authorino/0.log"
Jul 09 04:41:52.757835 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:52.757796    2577 patch_prober.go:28] interesting pod/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 04:41:52.758283 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:52.757855    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 04:41:55.396679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.396624    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:41:55.470265 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.470235    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-th7dt_1bc717cc-5e30-4221-b74f-89b63b207308/manager/0.log"
Jul 09 04:41:55.526050 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526022    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526113    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52v4w\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526184    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526219    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526269    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526302    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526346 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo\") pod \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\" (UID: \"c5a4613a-3cba-4352-9571-aa8f6e73c80a\") "
Jul 09 04:41:55.526639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526390    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:41:55.526639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526406    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:41:55.526639 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526420    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:41:55.526787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526682    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istiod-ca-cert\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.526787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526700    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-credential-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.526787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526714    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-socket\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.526787 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.526748    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:41:55.527352 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.527328    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:41:55.528579 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.528554    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:41:55.528867 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.528845    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w" (OuterVolumeSpecName: "kube-api-access-52v4w") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "kube-api-access-52v4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:55.528937 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.528910    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:41:55.528986 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.528929    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5a4613a-3cba-4352-9571-aa8f6e73c80a" (UID: "c5a4613a-3cba-4352-9571-aa8f6e73c80a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:41:55.585693 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.585670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l882p_61b9ad92-8594-477c-a5a6-708c2b6904fe/manager/0.log"
Jul 09 04:41:55.627087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627062    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52v4w\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-kube-api-access-52v4w\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.627087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627084    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-envoy\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.627251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627093    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-token\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.627251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627101    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-data\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.627251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627108    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5a4613a-3cba-4352-9571-aa8f6e73c80a-workload-certs\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.627251 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.627116    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5a4613a-3cba-4352-9571-aa8f6e73c80a-istio-podinfo\") on node \"ip-10-0-135-255.ec2.internal\" DevicePath \"\""
Jul 09 04:41:55.701157 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.701087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68d56bd64d-szkw4_63cc533b-e9b6-46d6-b461-6ef612bce631/manager/0.log"
Jul 09 04:41:55.805535 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:55.805511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f25h2_2fbf5964-e416-4540-a6fa-dc30ab7f24f4/server/0.log"
Jul 09 04:41:56.134510 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.134455    2577 generic.go:356] "Generic (PLEG): container finished" podID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerID="d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d" exitCode=0
Jul 09 04:41:56.134728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.134510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerDied","Data":"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"}
Jul 09 04:41:56.134728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.134531    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"
Jul 09 04:41:56.134728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.134587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn" event={"ID":"c5a4613a-3cba-4352-9571-aa8f6e73c80a","Type":"ContainerDied","Data":"8c2dfdc8902649909d9da58bb2f95667791ba2b7991415a339c2d0e68ac756b1"}
Jul 09 04:41:56.134728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.134619    2577 scope.go:117] "RemoveContainer" containerID="d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"
Jul 09 04:41:56.144880 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.144862    2577 scope.go:117] "RemoveContainer" containerID="46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"
Jul 09 04:41:56.154573 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.154549    2577 scope.go:117] "RemoveContainer" containerID="d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"
Jul 09 04:41:56.155240 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:56.155209    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d\": container with ID starting with d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d not found: ID does not exist" containerID="d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"
Jul 09 04:41:56.155360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.155248    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d"} err="failed to get container status \"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d\": rpc error: code = NotFound desc = could not find container \"d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d\": container with ID starting with d156c1eb96e5753362508dac7c783925ada7a6aff52911534348fb478bee900d not found: ID does not exist"
Jul 09 04:41:56.155360 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.155269    2577 scope.go:117] "RemoveContainer" containerID="46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"
Jul 09 04:41:56.155525 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.155432    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-8zh6m_f171cc86-f9e1-4760-928b-04d849fcdaa3/copy-manifests/0.log"
Jul 09 04:41:56.155597 ip-10-0-135-255 kubenswrapper[2577]: E0709 04:41:56.155561    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3\": container with ID starting with 46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3 not found: ID does not exist" containerID="46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"
Jul 09 04:41:56.155638 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.155607    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3"} err="failed to get container status \"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3\": rpc error: code = NotFound desc = could not find container \"46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3\": container with ID starting with 46227169a7aa2fe31a6387482e118d5133f1c86f8f8f4c5b8d9213244dfc43c3 not found: ID does not exist"
Jul 09 04:41:56.156307 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.156291    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:41:56.161489 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.161458    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-65509e-openshift-default-778f49855f-zj7gn"]
Jul 09 04:41:56.162145 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.162126    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-8zh6m_f171cc86-f9e1-4760-928b-04d849fcdaa3/manager/0.log"
Jul 09 04:41:56.608020 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:56.607994    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6dc7d55d4-px48k_e6afaf93-9b8f-46e5-9711-7b4bc8c000d5/maas-api/0.log"
Jul 09 04:41:57.180976 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.180945    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" path="/var/lib/kubelet/pods/c5a4613a-3cba-4352-9571-aa8f6e73c80a/volumes"
Jul 09 04:41:57.446691 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.446612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/pull/0.log"
Jul 09 04:41:57.452485 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.452438    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/extract/0.log"
Jul 09 04:41:57.459764 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.459746    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/util/0.log"
Jul 09 04:41:57.565411 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.565386    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/util/0.log"
Jul 09 04:41:57.571151 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.571134    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/pull/0.log"
Jul 09 04:41:57.576697 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.576662    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/extract/0.log"
Jul 09 04:41:57.684057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.684031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/util/0.log"
Jul 09 04:41:57.689834 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.689817    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/pull/0.log"
Jul 09 04:41:57.696153 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.696135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/extract/0.log"
Jul 09 04:41:57.814745 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.814670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/util/0.log"
Jul 09 04:41:57.821355 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.821335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/pull/0.log"
Jul 09 04:41:57.830168 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.830146    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/extract/0.log"
Jul 09 04:41:57.944082 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:57.944053    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8869bc4d-jt4kx_38e18490-3acd-4e52-b2b6-ca7ac1b3faeb/authorino/0.log"
Jul 09 04:41:58.064088 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:58.064060    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hmk86_1b8c1d34-b174-4002-897d-2b14cbae6c0d/manager/0.log"
Jul 09 04:41:58.167407 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:58.167385    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sd6h9_3ef83895-c441-4770-ae66-3176275a0a98/manager/0.log"
Jul 09 04:41:58.384165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:58.384117    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rkcwt_9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8/registry-server/0.log"
Jul 09 04:41:58.495960 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:58.495872    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dbmn2_b38f252c-3fcc-426d-8ea2-79f6fe3d378b/manager/0.log"
Jul 09 04:41:58.721271 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:58.721242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qjsfq_d5a13346-95df-4cd5-8821-077331e64d81/manager/0.log"
Jul 09 04:41:59.051568 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:59.051534    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787l6q4_5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8/istio-proxy/0.log"
Jul 09 04:41:59.275262 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:59.275233    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779b7dc9c4-hl5dr_20f62427-52fa-4201-ad10-ed3adf3e8dea/kube-auth-proxy/0.log"
Jul 09 04:41:59.630211 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:59.630168    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8fq2v_416930c5-8930-4dcf-b5ce-ec9154266a5e/payload-pre-processing/0.log"
Jul 09 04:41:59.743287 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:41:59.743248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nmzr6_1cce0859-a3ef-4fb4-8742-5d079b312cd1/payload-processing/0.log"
Jul 09 04:42:00.305057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:00.305031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-2qq9p_fe6653ea-3449-4f95-9ebb-33c4d7ff658e/main/0.log"
Jul 09 04:42:05.027063 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027028    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8skv/must-gather-cc6kc"]
Jul 09 04:42:05.027494 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027481    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ca528b-83ae-495c-91de-34cb4505f361" containerName="maas-api"
Jul 09 04:42:05.027543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027496    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ca528b-83ae-495c-91de-34cb4505f361" containerName="maas-api"
Jul 09 04:42:05.027543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027505    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" containerName="maas-api"
Jul 09 04:42:05.027543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027510    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" containerName="maas-api"
Jul 09 04:42:05.027543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027533    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027543 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027539    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027545    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027551    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027557    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9504263d-d346-4a80-aa63-34af7098c101" containerName="test-curl-7530-140361441258352"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027563    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9504263d-d346-4a80-aa63-34af7098c101" containerName="test-curl-7530-140361441258352"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027622    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ca528b-83ae-495c-91de-34cb4505f361" containerName="maas-api"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027630    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027637    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5a4613a-3cba-4352-9571-aa8f6e73c80a" containerName="istio-proxy"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027644    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9327b3c-d3d1-4ac3-81f0-3b3f3e71b859" containerName="maas-api"
Jul 09 04:42:05.027703 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.027650    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9504263d-d346-4a80-aa63-34af7098c101" containerName="test-curl-7530-140361441258352"
Jul 09 04:42:05.032688 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.032670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.034722 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.034687    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8skv\"/\"openshift-service-ca.crt\""
Jul 09 04:42:05.035440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.035421    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8skv\"/\"kube-root-ca.crt\""
Jul 09 04:42:05.035613 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.035543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v8skv\"/\"default-dockercfg-92sp8\""
Jul 09 04:42:05.038947 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.038925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/must-gather-cc6kc"]
Jul 09 04:42:05.051534 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.051509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:42:05.108165 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.108128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:42:05.116328 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.116304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5g4t\" (UniqueName: \"kubernetes.io/projected/dd829dc7-40b9-40e2-80c0-40dce054a5fc-kube-api-access-j5g4t\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.116425 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.116361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dd829dc7-40b9-40e2-80c0-40dce054a5fc-must-gather-output\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.217939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.217907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5g4t\" (UniqueName: \"kubernetes.io/projected/dd829dc7-40b9-40e2-80c0-40dce054a5fc-kube-api-access-j5g4t\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.218110 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.217961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dd829dc7-40b9-40e2-80c0-40dce054a5fc-must-gather-output\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.218282 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.218265    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dd829dc7-40b9-40e2-80c0-40dce054a5fc-must-gather-output\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.228431 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.228409    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5g4t\" (UniqueName: \"kubernetes.io/projected/dd829dc7-40b9-40e2-80c0-40dce054a5fc-kube-api-access-j5g4t\") pod \"must-gather-cc6kc\" (UID: \"dd829dc7-40b9-40e2-80c0-40dce054a5fc\") " pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.343953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.343865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/must-gather-cc6kc"
Jul 09 04:42:05.476324 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:05.476295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/must-gather-cc6kc"]
Jul 09 04:42:05.477752 ip-10-0-135-255 kubenswrapper[2577]: W0709 04:42:05.477725    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd829dc7_40b9_40e2_80c0_40dce054a5fc.slice/crio-367a5385e927a9e1552ef46dc8fe49cf9f2ec29dfef3beff4bd6f7e7943d55c3 WatchSource:0}: Error finding container 367a5385e927a9e1552ef46dc8fe49cf9f2ec29dfef3beff4bd6f7e7943d55c3: Status 404 returned error can't find the container with id 367a5385e927a9e1552ef46dc8fe49cf9f2ec29dfef3beff4bd6f7e7943d55c3
Jul 09 04:42:06.190890 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:06.190859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/must-gather-cc6kc" event={"ID":"dd829dc7-40b9-40e2-80c0-40dce054a5fc","Type":"ContainerStarted","Data":"367a5385e927a9e1552ef46dc8fe49cf9f2ec29dfef3beff4bd6f7e7943d55c3"}
Jul 09 04:42:07.200266 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:07.200231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/must-gather-cc6kc" event={"ID":"dd829dc7-40b9-40e2-80c0-40dce054a5fc","Type":"ContainerStarted","Data":"c052f7ea8d681e9f9b2d500a906d8133a426cc44dd7cbe2126e6da28ea18d62a"}
Jul 09 04:42:07.200819 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:07.200795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/must-gather-cc6kc" event={"ID":"dd829dc7-40b9-40e2-80c0-40dce054a5fc","Type":"ContainerStarted","Data":"ac9f661212b7631a83c6e1ca517f3f41852ba2bb56819746e3f05c6504241a57"}
Jul 09 04:42:07.973776 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:07.973744    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9cr6q_6dcb5037-c133-44cf-9617-ec0a07a55301/global-pull-secret-syncer/0.log"
Jul 09 04:42:08.030091 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:08.030057    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log"
Jul 09 04:42:08.140496 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:08.140433    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-255.ec2.internal_ed53264ef20f88c821b678c86f72e5cd/haproxy/0.log"
Jul 09 04:42:12.577139 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.577093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/extract/0.log"
Jul 09 04:42:12.604684 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.604652    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/util/0.log"
Jul 09 04:42:12.627487 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.627441    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g4xv9_ae4a3c01-b668-401d-a195-d486ffdc1b22/pull/0.log"
Jul 09 04:42:12.658821 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.658791    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/extract/0.log"
Jul 09 04:42:12.679783 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.679750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/util/0.log"
Jul 09 04:42:12.704030 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.703164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb52z_db812661-3716-4089-a69a-845bad3b0af1/pull/0.log"
Jul 09 04:42:12.729036 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.729003    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/extract/0.log"
Jul 09 04:42:12.754106 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.754036    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/util/0.log"
Jul 09 04:42:12.775718 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.775672    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p5j9n_c389c884-7b60-4d13-82e4-ff110ce95d0f/pull/0.log"
Jul 09 04:42:12.807647 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.807613    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/extract/0.log"
Jul 09 04:42:12.829602 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.829488    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/util/0.log"
Jul 09 04:42:12.856581 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:12.856544    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j67gx_05876de0-f871-48f9-85b2-eaf12b7b9541/pull/0.log"
Jul 09 04:42:13.236775 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.236742    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8869bc4d-jt4kx_38e18490-3acd-4e52-b2b6-ca7ac1b3faeb/authorino/0.log"
Jul 09 04:42:13.273016 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.272985    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hmk86_1b8c1d34-b174-4002-897d-2b14cbae6c0d/manager/0.log"
Jul 09 04:42:13.309641 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.309610    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sd6h9_3ef83895-c441-4770-ae66-3176275a0a98/manager/0.log"
Jul 09 04:42:13.379893 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.379854    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rkcwt_9b8cc5f2-b8f8-4aa6-8282-266f6f4aecb8/registry-server/0.log"
Jul 09 04:42:13.576531 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.576413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dbmn2_b38f252c-3fcc-426d-8ea2-79f6fe3d378b/manager/0.log"
Jul 09 04:42:13.682488 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:13.681447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qjsfq_d5a13346-95df-4cd5-8821-077331e64d81/manager/0.log"
Jul 09 04:42:15.347502 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.347454    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/alertmanager/0.log"
Jul 09 04:42:15.371671 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.371643    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/config-reloader/0.log"
Jul 09 04:42:15.394953 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.394901    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/kube-rbac-proxy-web/0.log"
Jul 09 04:42:15.419512 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.419478    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/kube-rbac-proxy/0.log"
Jul 09 04:42:15.443392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.443364    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/kube-rbac-proxy-metric/0.log"
Jul 09 04:42:15.468253 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.468229    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/prom-label-proxy/0.log"
Jul 09 04:42:15.490756 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.490684    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12eed2a2-f8a8-4556-98cd-84abfb5d0e32/init-config-reloader/0.log"
Jul 09 04:42:15.718992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.718960    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-6fvb4_07e5f30a-e6c4-485d-85a5-d15885e27d88/kube-state-metrics/0.log"
Jul 09 04:42:15.739176 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.739141    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-6fvb4_07e5f30a-e6c4-485d-85a5-d15885e27d88/kube-rbac-proxy-main/0.log"
Jul 09 04:42:15.763484 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.763370    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-6fvb4_07e5f30a-e6c4-485d-85a5-d15885e27d88/kube-rbac-proxy-self/0.log"
Jul 09 04:42:15.944499 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.944444    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p4gpk_ff71642a-d812-44ad-b774-38d253a9c03e/node-exporter/0.log"
Jul 09 04:42:15.965402 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.965371    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p4gpk_ff71642a-d812-44ad-b774-38d253a9c03e/kube-rbac-proxy/0.log"
Jul 09 04:42:15.988184 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:15.988152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p4gpk_ff71642a-d812-44ad-b774-38d253a9c03e/init-textfile/0.log"
Jul 09 04:42:16.090019 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.089932    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mnzc2_b209ea7b-6073-42b4-aaeb-679f5df5c42c/kube-rbac-proxy-main/0.log"
Jul 09 04:42:16.113246 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.113217    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mnzc2_b209ea7b-6073-42b4-aaeb-679f5df5c42c/kube-rbac-proxy-self/0.log"
Jul 09 04:42:16.144434 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.144392    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mnzc2_b209ea7b-6073-42b4-aaeb-679f5df5c42c/openshift-state-metrics/0.log"
Jul 09 04:42:16.448219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.448187    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c5d9988d9-9j2td_875f2d19-c79e-4f9b-8363-139af5d25b0c/telemeter-client/0.log"
Jul 09 04:42:16.477337 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.477264    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c5d9988d9-9j2td_875f2d19-c79e-4f9b-8363-139af5d25b0c/reload/0.log"
Jul 09 04:42:16.501573 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.501542    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c5d9988d9-9j2td_875f2d19-c79e-4f9b-8363-139af5d25b0c/kube-rbac-proxy/0.log"
Jul 09 04:42:16.713440 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:16.713341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5vfb_ca467583-1e69-4d74-8c16-e5e033f74b6e/konnectivity-agent/0.log" limit=1024
Jul 09 04:42:17.828420 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:17.828392    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-z2jn5_04793198-4033-4011-bebf-d70b4b2b8185/networking-console-plugin/0.log"
Jul 09 04:42:17.954031 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:17.953954    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v8skv/must-gather-cc6kc" podStartSLOduration=12.089372799 podStartE2EDuration="12.953934238s" totalImagesPullingTime="864.561439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:42:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:07.217314094 +0000 UTC m=+2526.491692003" watchObservedRunningTime="2026-07-09 04:42:17.953934238 +0000 UTC m=+2537.228312146"
Jul 09 04:42:17.956923 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:17.956895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"]
Jul 09 04:42:17.965916 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:17.965886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:17.969219 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:17.969194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"]
Jul 09 04:42:18.058192 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.058160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-podres\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.058399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.058218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-lib-modules\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.058399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.058256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-sys\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.058399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.058273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfm4b\" (UniqueName: \"kubernetes.io/projected/eb6ab173-5506-4968-8e67-334d4321d4f8-kube-api-access-zfm4b\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.058399 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.058319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-proc\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159290 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-proc\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-podres\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-lib-modules\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159389    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-proc\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-sys\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159520 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfm4b\" (UniqueName: \"kubernetes.io/projected/eb6ab173-5506-4968-8e67-334d4321d4f8-kube-api-access-zfm4b\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159530    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-lib-modules\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-podres\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.159728 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.159558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb6ab173-5506-4968-8e67-334d4321d4f8-sys\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.167380 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.167317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfm4b\" (UniqueName: \"kubernetes.io/projected/eb6ab173-5506-4968-8e67-334d4321d4f8-kube-api-access-zfm4b\") pod \"perf-node-gather-daemonset-nbzmz\" (UID: \"eb6ab173-5506-4968-8e67-334d4321d4f8\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.280414 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.280383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:18.469600 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.468734    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"]
Jul 09 04:42:18.529094 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.528829    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/2.log"
Jul 09 04:42:18.534924 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:18.534816    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2vxxm_d7de353e-7173-43ab-879e-97b27640cbc6/console-operator/3.log"
Jul 09 04:42:19.025087 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:19.025062    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-87d9v_cbcf6b76-9a1f-4961-a062-4f0eb81c3a72/download-server/0.log"
Jul 09 04:42:19.304831 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:19.304743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz" event={"ID":"eb6ab173-5506-4968-8e67-334d4321d4f8","Type":"ContainerStarted","Data":"b6861aeef37d05c9fd870b05d08b8d0cb881de0ec4cb277b1ecb99e910e60fa6"}
Jul 09 04:42:19.304831 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:19.304815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz" event={"ID":"eb6ab173-5506-4968-8e67-334d4321d4f8","Type":"ContainerStarted","Data":"4d718c11cf9434b43acc841a1a72363c8d9880741040f293846de7b277d13885"}
Jul 09 04:42:19.305949 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:19.305921    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:19.323818 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:19.323772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz" podStartSLOduration=2.323756064 podStartE2EDuration="2.323756064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:19.322758514 +0000 UTC m=+2538.597136422" watchObservedRunningTime="2026-07-09 04:42:19.323756064 +0000 UTC m=+2538.598133972"
Jul 09 04:42:20.648857 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:20.648827    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wwrtx_c4538ad5-df7b-497a-b315-8b49ced9d705/dns/0.log"
Jul 09 04:42:20.682006 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:20.681980    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wwrtx_c4538ad5-df7b-497a-b315-8b49ced9d705/kube-rbac-proxy/0.log"
Jul 09 04:42:20.759392 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:20.759368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xbljd_29bad720-862e-46d4-a5e6-39b7fe607ff6/dns-node-resolver/0.log"
Jul 09 04:42:21.275853 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:21.275809    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d5cf85f5f-4bkx5_86fc61f0-2850-482e-8cf4-64a3fbc63529/registry/0.log"
Jul 09 04:42:21.347461 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:21.347434    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xmpxf_c8d6e6e9-c491-40bf-b405-df699fc273b6/node-ca/0.log"
Jul 09 04:42:22.291316 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:22.291290    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787l6q4_5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8/istio-proxy/0.log"
Jul 09 04:42:22.510049 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:22.510014    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779b7dc9c4-hl5dr_20f62427-52fa-4201-ad10-ed3adf3e8dea/kube-auth-proxy/0.log"
Jul 09 04:42:22.679948 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:22.679909    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8fq2v_416930c5-8930-4dcf-b5ce-ec9154266a5e/payload-pre-processing/0.log"
Jul 09 04:42:22.751832 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:22.751800    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nmzr6_1cce0859-a3ef-4fb4-8742-5d079b312cd1/payload-processing/0.log"
Jul 09 04:42:23.488066 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:23.488043    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bf7hv_4eff9c62-944c-454a-8dd3-830e3fcc7c23/serve-healthcheck-canary/0.log"
Jul 09 04:42:24.233422 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:24.233396    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlt7p_23a60a31-a3f7-44ae-b5bc-49d8bc0f3516/kube-rbac-proxy/0.log"
Jul 09 04:42:24.254991 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:24.254963    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlt7p_23a60a31-a3f7-44ae-b5bc-49d8bc0f3516/exporter/0.log"
Jul 09 04:42:24.276809 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:24.276784    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlt7p_23a60a31-a3f7-44ae-b5bc-49d8bc0f3516/extractor/0.log"
Jul 09 04:42:26.327939 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:26.327908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-nbzmz"
Jul 09 04:42:26.406949 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:26.406923    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-th7dt_1bc717cc-5e30-4221-b74f-89b63b207308/manager/0.log"
Jul 09 04:42:26.577679 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:26.577634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l882p_61b9ad92-8594-477c-a5a6-708c2b6904fe/manager/0.log"
Jul 09 04:42:26.710355 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:26.710318    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68d56bd64d-szkw4_63cc533b-e9b6-46d6-b461-6ef612bce631/manager/0.log"
Jul 09 04:42:26.752102 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:26.752072    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f25h2_2fbf5964-e416-4540-a6fa-dc30ab7f24f4/server/0.log"
Jul 09 04:42:27.060498 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:27.060414    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-8zh6m_f171cc86-f9e1-4760-928b-04d849fcdaa3/manager/0.log"
Jul 09 04:42:27.086811 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:27.086777    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-8zh6m_f171cc86-f9e1-4760-928b-04d849fcdaa3/copy-manifests/0.log"
Jul 09 04:42:27.891204 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:27.891148    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787l6q4_5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8/istio-proxy/0.log"
Jul 09 04:42:28.092948 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:28.092916    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779b7dc9c4-hl5dr_20f62427-52fa-4201-ad10-ed3adf3e8dea/kube-auth-proxy/0.log"
Jul 09 04:42:28.233086 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:28.233054    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8fq2v_416930c5-8930-4dcf-b5ce-ec9154266a5e/payload-pre-processing/0.log"
Jul 09 04:42:28.289895 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:28.289331    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nmzr6_1cce0859-a3ef-4fb4-8742-5d079b312cd1/payload-processing/0.log"
Jul 09 04:42:29.691269 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:29.691235    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787l6q4_5bc7f34b-6f3f-4fad-b419-8c70fd66cfa8/istio-proxy/0.log"
Jul 09 04:42:29.887521 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:29.887487    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779b7dc9c4-hl5dr_20f62427-52fa-4201-ad10-ed3adf3e8dea/kube-auth-proxy/0.log"
Jul 09 04:42:30.036637 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:30.036562    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8fq2v_416930c5-8930-4dcf-b5ce-ec9154266a5e/payload-pre-processing/0.log"
Jul 09 04:42:30.091966 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:30.091934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nmzr6_1cce0859-a3ef-4fb4-8742-5d079b312cd1/payload-processing/0.log"
Jul 09 04:42:34.754782 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.754755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/kube-multus-additional-cni-plugins/0.log"
Jul 09 04:42:34.775699 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.775669    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/egress-router-binary-copy/0.log"
Jul 09 04:42:34.795873 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.795855    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/cni-plugins/0.log"
Jul 09 04:42:34.816749 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.816730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/bond-cni-plugin/0.log"
Jul 09 04:42:34.839329 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.839310    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/routeoverride-cni/0.log"
Jul 09 04:42:34.860865 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.860842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/whereabouts-cni-bincopy/0.log"
Jul 09 04:42:34.881175 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:34.881158    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-88f9s_c1c97c43-619e-492d-8e51-7ec49f54eaea/whereabouts-cni/0.log"
Jul 09 04:42:35.231961 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:35.231935    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fltq9_de1a7fdc-d6bf-4bdd-8c62-f9239d56c64a/kube-multus/0.log"
Jul 09 04:42:35.374130 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:35.374098    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-st6ww_ba58ffef-ea23-486c-ad2e-be920dc8173c/network-metrics-daemon/0.log"
Jul 09 04:42:35.397140 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:35.397120    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-st6ww_ba58ffef-ea23-486c-ad2e-be920dc8173c/kube-rbac-proxy/0.log"
Jul 09 04:42:36.668181 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.668156    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/ovn-controller/0.log"
Jul 09 04:42:36.687057 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.687030    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/ovn-acl-logging/0.log"
Jul 09 04:42:36.712547 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.712498    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/kube-rbac-proxy-node/0.log"
Jul 09 04:42:36.735099 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.735077    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 04:42:36.760992 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.760970    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/northd/0.log"
Jul 09 04:42:36.783552 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.783534    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/nbdb/0.log"
Jul 09 04:42:36.804386 ip-10-0-135-255 kubenswrapper[2577]: I0709 04:42:36.804368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jchmt_83bbe97f-cc76-4bb6-81f6-a8abbe5ada48/sbdb/0.log"